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The  following  nonlinear  assignment  problems  (NAPs)  are  examined  in  this  dis- 
sertation: quadratic  assignment  problem  (QAP),  biquadratic  assignment  problem 
(BiQAP),  turbine  balancing  problem  (TBP),  and  multidimensional  assignment 
problem  (MAP).  All  of  the  above  are  NP-hard  combinatorial  optimization  prob- 
lems that  are  natural  extensions  of  the  classical  linear  assignment  problem.  NAPs 
have  numerous  applications  in  different  fields,  ranging  from  the  assignment  of  fa- 
cilities to  sites  in  location  theory,  to  the  tracking  of  elementary  particles  in  high 
energy  physics.  In  this  dissertation  we  present  heuristic  algorithms  for  solving  the 
above  NAPs. 

The  dissertation  is  divided  into  three  parts.  In  the  first  part,  a survey  of  nonlin- 
ear assignment  problems  is  presented,  with  focus  on  the  most  studied,  QAP.  Differ- 
ent formulations,  complexity,  lower  bounds,  exact  and  heuristic  solution  methods, 
and  essentially  all  aspects  that  have  been  studied  for  these  problems  are  presented. 

The  second  part  presents  heuristic  algorithms  for  solving  sparse  QAP  instances, 
the  BiQAP  and  the  TBP.  Sparse  QAP  instances  are  those  whose  input  matrices 
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contain  a large  fraction  of  zeroes.  We  present  a heuristic  algorithm  that  exploits 
the  sparsity  of  such  instances,  and  provides  approximate  solutions.  The  TBP  is 
formulated  as  a QAP,  and  an  algorithm  that  outperforms  the  current  industry 
practice  is  presented.  The  BiQAP  is  an  assignment  problem  where  the  objective 
function  is  a fourth-degree  multivariable  polynomial.  We  propose  a heuristic  al- 
gorithm for  solving  the  BiQAP,  and  computational  results  on  instances  described 
in  the  literature  indicate  that  it  consistently  finds  better  solutions  than  previously 
described  algorithms. 

In  the  third  part,  heuristic  algorithms  for  solving  the  MAP  and  its  special  case 
the  three-dimensional  assignment  problem  are  presented.  The  multi-target  multi- 
sensor tracking  problem  (MTMST)  which  has  applications  in  military  tracking 
systems  and  in  high  energy  physics  is  examined,  and  we  show  how  the  MTMST 
can  be  formulated  as  an  MAP.  A heuristic  algorithm  that  solves  the  MAP  is 
described,  and  it  is  tested  on  a set  of  problems  provided  by  the  US  Air  Force. 

Computational  results  indicate  that  all  of  the  suggested  algorithms  are  among 
the  best  in  the  literature  in  terms  of  solution  quality  and  computational  time. 
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CHAPTER  1 
INTRODUCTION 


1.1  Nonlinear  Assignment  Problems 

We  begin  this  introductory  section  with  the  most  basic  of  all  assignment  prob- 
lems, the  linear  assignment  problem  (LAP).  Consider  that  we  are  given  n facilities 
and  locations,  and  a matrix  IR"^”  9 (7  :=  (cij)  where  represents  the  cost  of 
placing  facility  i to  location  j.  Our  objective  is  to  find  the  optimal  assignment  of 
facilities  to  locations  such  that  the  total  placement  cost  is  minimized.  Using  the 
decision  variables  Xij  G {0, 1}  as 

/ 

1 if  facility  i is  assigned  to  location  j, 

Xij  — ^ 

0 otherwise, 

we  can  formulate  the  LAP  as  an  integer  0 — 1 program 

n 

min  2^  CijXij 
ij=l 

n 

s.t.  2^Xij  = l,  j = l,2,...,n, 

i=l 

n 

z = 1, 2, . . . , n, 
j=i 

Xij  G {0,1},  i,j  = l,2,...,n. 

Equivalently  using  permutations  we  can  formulate  the  LAP  as 


1 


2 


n 


min 


where  ri;v  is  the  set  of  all  permutations  p : N N,  and  N — {1,2,...  , n}.  The 
LAP  is  a classical  optimization  problem  and  can  be  solved  very  efficiently  [6]. 

Nonlinear  assignment  problems  (NAPs)  are  extensions  of  the  LAP,  and  they 
can  be  divided  into  two  major  classes,  M-adic  assignment  problems  introduced  by 
Lawler  [123],  and  multidimensional  assignment  problems  (MPAs)  introduced  by 
Pierskalla  [160]. 

In  the  M-adic  assignment  problem  class,  the  dimension  of  the  cost  array  in 
(1.1)  becomes  2M,  M > 2,  and  we  permute  over  M indices  of  C in  the  objective 
function.  For  example  the  3-adic  assignment  problem  can  be  formulated  as 


n n n 


min 


'ijkp{i)p{j)p{k)i 


i=\  j=i  fc=i 


where  C :=  (cijkims)  ^ IR"^^,  M = 3.  In  a similar  way  we  can  define  the  quadratic 
(M  = 2),  and  the  biquadratic  (M  = 4)  assignment  problems.  In  the  MAP  class, 
the  dimension  of  the  cost  array  in  (1.1)  becomes  M > 3,  and  we  sum  over  n values 
of  C.  However  C is  permuted  over  M — 1 indices  by  different  permutations.  For 
example  the  three  dimensional  assignment  problem  can  be  formulated  as 


n 


min 


2=1 


where  C :=  {cijk)  ^ = 3.  In  chapter  6 we  will  describe  an  application 

involving  the  general  formulation  of  the  MAP,  where  M itself  a variable  of  the 
input  data. 
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1.2  Main  Contributions  and  Organization  of  the  Dissertation 

Our  contributions  can  be  summarized  as  follows.  In  chapter  3 we  present  a 
heuristic  algorithm  for  solving  sparse  quadratic  assignment  problem  (QAP)  in- 
stances. Computational  results  indicate  that  our  algorithm  is  competitive  with 
respect  to  other  algorithms  in  the  literature,  and  its  computational  time  is  in- 
versely proportional  to  the  sparsity  of  the  problem.  In  chapter  5 a heuristic  algo- 
rithm for  solving  the  biquadratic  assignment  problem  (BiQAP)  is  presented.  Our 
algorithm  finds  the  optimal  solution  for  every  problem  in  a set  of  instances  with 
known  optimal  solutions  taken  from  the  literature.  The  turbine  balancing  problem 
is  formulated  as  a special  case  of  the  QAP  in  chapter  4,  and  a heuristic  algorithm 
is  presented  for  solving  the  resulting  problem.  Computational  results  demonstrate 
that  the  proposed  algorithm  outperforms  the  current  industry  practice.  Finally  in 
chapter  6 we  present  heuristic  algorithms  for  solving  the  MAP  and  its  special  case 
the  3-dimensional  assignment  Problem.  We  show  how  the  multi  target  multi  sensor 
tracking  (MTMST)  problem  can  be  modeled  as  an  optimization  problem,  which 
is  equivalent  to  the  MAP.  Computational  results  on  a set  of  problems  provided  by 
the  US  Air  Force  show  that  our  algorithm  produces  good  quality  solutions  in  real 
time. 

The  dissertation  is  organized  in  the  following  manner.  The  material  presented 
in  chapter  2 is  a comprehensive  survey  on  nonlinear  assignment  problem  with  focus 
on  the  QAP,  and  can  serve  as  an  introductory  chapter.  Different  formulations, 
complexity,  lower  bounds,  exact  and  heuristic  solution  methods,  and  essentially 
every  aspect  that  has  been  studied  for  NAPs  is  presented. 

In  chapter  3,  FORTRAN  subroutines  for  finding  approximate  solutions  of 
sparse  instances  of  the  QAP  using  a greedy  randomized  adaptive  search  proce- 
dure (GRASP)  are  presented.  The  design  and  implementation  of  the  code  are 
described  in  detail.  Computational  results  comparing  the  new  subroutines  with  a 
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dense  version  of  the  code  in  Li  et  al.  [129]  show  that  the  speedup  increases  with 
the  sparsity  of  the  data. 

In  chapter  4 the  turbine  balancing  problem  is  presented,  which  is  a combinato- 
rial optimization  problem  arising  in  the  manufacturing  and  maintenance  of  turbine 
engines.  The  TBP  is  formulated  as  a QAP,  and  a heuristic  algorithm  is  proposed 
for  solving  the  resulting  problem.  Computational  results  were  performed  on  a set 
of  problems  provided  by  Pratt  & Whitney,  and  from  instances  taken  from  the  lit- 
erature. The  proposed  algorithm  outperforms  the  current  practice  used  for  solving 
the  TBP,  and  has  the  best  overall  performance  with  respect  to  other  heuristic 
algorithms  in  the  literature. 

Chapter  5 is  concerned  with  the  biquadratic  assignment  problem.  The  BiQAP 
is  a generalization  of  the  quadratic  assignment  problem.  It  is  a nonlinear  integer 
programming  problem  where  the  objective  function  is  a fourth-degree  multivariable 
polynomial  and  the  feasible  domain  is  the  assignment  polytope.  BiQAP  problems 
appear  in  VLSI  synthesis.  Due  to  the  difficulty  of  this  problem,  only  heuristic 
solution  approaches  have  been  proposed  in  the  literature.  A GRASP  for  solv- 
ing the  BiQAP  is  presented,  and  computational  results  on  instances  described  in 
the  literature  indicate  that  this  procedure  consistently  finds  better  solutions  than 
previously  described  algorithms. 

In  sections  6.1  through  6.6  of  chapter  6,  a GRASP  is  presented  for  comput- 
ing approximate  solutions  to  the  multitarget  multisensor  tracking  problem.  The 
MTMST  problem  may  be  described  as  follows:  Given  a large  number  of  sensors, 
each  with  some  error,  reporting  a large  number  of  closely  spaced  objects  at  dis- 
crete intervals  in  time,  determine  trajectory  estimates  for  any  targets  that  may 
be  present.  At  any  single  interval  in  time,  the  MTMST  is  shown  to  be  a multidi- 
mensional assignment  problem.  Since  the  objective  is  to  select  a target  hypothesis 
and  partition  of  the  measurements  that  is  “most  likely”  to  occur,  a likelihood  cost 
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function  and  partitioning  constraint  set  are  developed.  Filtering  techniques  are 
established  which  promise  to  dramatically  decrease  the  cardinality  of  the  feasible 
partition  space,  enabling  a GRASP  heuristic  to  be  applied  effectively.  Finally  in 
section  6.7,  a GRASP  is  presented  for  solving  the  three-dimensional  assignment 
problem,  and  computational  experiments  on  a set  of  instances  taken  from  the 
literature  are  reported. 


CHAPTER  2 

THE  QUADRATIC  ASSIGNMENT  PROBLEM 


2.1  Introduction 

The  quadratic  assignment  problem  (QAP)  was  introduced  by  Koopmans  and 
Beckmann  in  1957  as  a mathematical  model  for  the  location  of  a set  of  indivisible 
economical  activities  [117].  Consider  the  problem  of  allocating  a set  of  facilities  to  a 
set  of  locations,  with  the  cost  being  a function  of  the  distance  and  flow  between  the 
facilities,  plus  costs  associated  with  a facility  being  placed  at  a certain  location. 
The  objective  is  to  assign  each  facility  to  a location  such  that  the  total  cost  is 
minimized.  Specifically,  we  are  given  three  nxn  input  matrices  with  real  elements 
F = (/ij),  D = (dki)  and  B = (bik),  where  fij  is  the  flow  between  the  facility  i and 
facility  j,  dki  is  the  distance  between  the  location  k and  location  I,  and  bik  is  the 
cost  of  placing  facility  i at  location  k.  The  Koopmans-Beckmann  version  of  the 
QAP  can  be  formulated  as  follows;  Let  n be  the  number  of  facilities  and  locations 
and  denote  by  N the  set  A^  = {1,2,...  ,n}.  The  QAP  is  then  formulated  as 


min 

peUN 


n 


n 


n 


ip{i) 


i=l  j=l 


i=l 


where  H^r  is  the  set  of  all  permutations  p : N ^ N.  Each  individual  product 
fijdp{i)p{j)  is  the  cost  of  assigning  facility  i to  location  p{i)  and  facility  j to  location 
p{j).  In  the  context  of  facility  location  the  matrices  F and  D are  symmetric  with 
zeros  in  the  diagonal,  and  all  the  matrices  are  nonnegative.  An  instance  of  a QAP 
with  input  matrices  F,D  and  B will  be  denoted  by  QAP{F,D,B),  while  we  will 
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denote  an  instance  by  QAP{F,  D),  if  there  is  no  linear  term  (i.e.,  B = 0).  A more 
general  version  of  the  QAP  was  introduced  by  Lawler  [123].  In  this  version  we 
are  given  a four-dimensional  array  C = {cijki)  of  coefficients  instead  of  the  two 
matrices  F and  D and  the  problem  can  be  stated  as 

n n n 

i|dn  "*■  hp{i)-  (2-2) 

^ i=l  j=l  i=l 

Clearly,  a Koopmans-Beckmann  problem  QAP{F,D,B)  can  be  formulated  as  a 
Lawler  QAP  by  setting  Cijki  :=  fijdki  for  all  with  i ^ j ox  k ^ I and 

Ciikk  '■=  fiidkk  + bik,  Otherwise.  Although  extensive  research  has  been  done  for 
more  than  three  decades,  the  QAP,  in  contrast  with  its  linear  counterpart  the  linear 
assignment  problem  (LAP),  remains  one  of  the  hardest  optimization  problems  and 
no  exact  algorithm  can  solve  problems  of  size  n > 20  in  reasonable  computational 
time.  In  fact,  Sahni  and  Gonzalez  [174]  have  shown  that  the  QAP  is  NP-hard  and 
that  even  finding  an  approximate  solution  within  some  constant  factor  from  the 
optimal  solution  cannot  be  done  in  polynomial  time  unless  P=NP.  These  results 
hold  even  for  the  Koopmans-Beckmann  QAP  with  coefficient  matrices  fulfilling  the 
triangle  inequality  (see  Queyranne  [165]).  So  far  only  for  a very  special  case  of  the 
Koopmans-Beckmann  QAP,  the  dense  linear  arrangement  problem,  a polynomial 
time  approximation  scheme  has  been  found  by  Arora,  Frieze,  and  Kaplan  [10]. 
Complexity  aspects  of  the  QAP  will  be  discussed  in  more  detail  in  §2.3. 

Let  us  conclude  this  section  with  a brief  review  of  some  of  the  many  appli- 
cations of  the  QAP.  In  addition  to  facility  layout  problems,  the  QAP  appears 
in  applications  such  as  backboard  wiring,  computer  manufacturing,  scheduling, 
process  communications,  turbine  balancing,  and  many  others.  One  of  the  earlier 
applications  goes  back  to  Steinberg  [179]  and  concerns  backboard  wiring.  Different 
devices  such  as  controls  and  displays  have  to  be  placed  on  a panel,  where  they 
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have  to  be  connected  to  each  other  by  wires.  The  problem  is  to  find  a positioning 
of  the  devices  so  as  to  minimize  the  total  wire  length.  Let  n be  the  number  of 
devices  to  be  placed  and  let  dki  denote  the  wire  length  from  position  k to  position 
1.  The  fiow  matrix  F = (fij)  is  given  by 


if  device  i is  connected  to  device  j, 
otherwise. 


Then  the  solution  to  the  corresponding  QAP  will  minimize  the  total  wire  length. 
Another  application  in  the  context  of  location  theory  is  a campus  planning  problem 
due  to  Dickey  and  Hopkins  [58].  The  problem  consists  of  planning  the  sites  of  n 
buildings  in  a campus,  where  dki  is  the  distance  from  site  k to  site  I,  and  fij  is  the 
traffic  intensity  between  building  i and  building  j The  objective  is  to  minimize  the 
total  walking  distance  between  the  buildings.  In  the  field  of  ergonomics  Burkard 
and  Offermann  [43]  showed  that  QAPs  can  be  applied  to  typewriter  keyboard  design. 
The  problem  is  to  arrange  the  keys  in  a keyboard  such  as  to  minimize  the  time 
needed  to  write  some  text.  Let  the  set  of  integers  N = {1,2, .. . ,n}  denote  the 
set  of  symbols  to  be  arranged.  Then  fij  denotes  the  frequency  of  the  appearance 
of  the  pair  of  symbols  i and  j.  The  entries  of  the  distance  matrix  D = dki  are  the 
times  needed  to  press  the  key  in  position  / after  pressing  the  key  in  position  k for 
all  the  keys  to  be  assigned.  Then  a permutation  p G Hat  describes  an  assignment 
of  symbols  to  keys.  An  optimal  solution  p*  for  the  QAP  minimizes  the  average 
time  for  writing  a text.  A similar  application  related  to  ergonomic  design,  is  the 
development  of  control  boards  in  order  to  minimize  eye  fatigue  by  McCormick  [136]. 
There  are  also  numerous  other  applications  of  the  QAP  in  different  fields,  for 
example,  hospital  layout  (Elshafei  [65]),  ranking  of  archaeological  data  (Krarup  and 
Pruzan  [119]),  ranking  of  a team  in  a relay  race  (Heffley  [96]),  scheduling  parallel 
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production  lines  (Geoffrion  and  Graves  [80]),  and  analyzing  chemical  reactions  for 
organic  compounds  (Ugi,  Bauer,  Friedrich,  Gasteiger,  Jochum,  and  Schubert  [184]). 


2.2  Formulations 


For  most  combinatorial  optimization  problems,  different  mathematical  formula- 
tions exist  that  are  equivalent.  These  formulations  reflect  the  efforts  of  researchers 
to  view  the  structural  characteristics  of  the  problem  from  a different  angle,  in 
hope  of  developing  new  solution  techniques.  The  different  formulations  that  will 
be  discussed  in  this  section  are  motivated  by  different  applications  or  from  different 
solution  techniques  that  appeared  in  the  literature.  Their  presentation  serves  two 
purposes.  First,  discussions  about  different  relaxations  and  lower  bounds  for  the 
QAP  in  later  sections  will  be  based  on  some  of  the  formulations  presented  here, 
and  secondly,  someone  with  little  or  no  acquaintance  with  the  QAP  will  develop  a 
good  insight  for  the  problem  after  reading  this  section.  In  the  formulation  (2.1), 
we  can  associate  with  each  p € fl^v  a 0-1  matrix  X = (xij)nxn,  such  that 


if  p(i)  = i, 
otherwise. 


In  other  words  the  elements  of  X must 


n 


j=l 


Xij  G ■{  0 , 1 } , 

which  define  the  vertices  of  the  well 


satisfy  the  following  assignment  constraints 

3 f ) 2, . . . , n, 

z = 1,  2, . . . , n, 
i,  j 1,2,...  , zz, 

known  assignment  polytope  (other  names 


include:  the  Birkhoff  polytope,  the  transportation  polytope,  the  perfect  matching 
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polytope  of  Kn,n,  etc.).  The  matrix  X is  also  called  a permutation  matrix,  and 
we  denote  the  set  of  all  such  permutation  matrices  by  X„  C {0,  Observe 

that  there  is  one-to-one  correspondence  between  and  X„,  and  the  cardinality 
of  both  sets  in  n\. 

2.2.1  Quadratic  0-1  Integer  Formulation 

Koopmans  and  Beckmann  [117]  introduced  the  QAP  as  a quadratic  0-1  integer 
program.  A solution  to  QAP{F,  D)  is  represented  by  a permutation  matrix  X = 
(xij)  where 


if  facility  i is  assigned  to  location  j, 
otherwise. 


The  problem  can  be  represented  then  as  an  integer  program  with  a quadratic 
objective  function  (hence  the  name  quadratic  assignment  problem)  as  follows 


min 


s.t. 


n n n n 


n 


2=1  j=l  k=l  l=l 


. .rp  . . 


n 


2 = 1 


n 

Xij  = 1,  z = 1, 2, . . . , n, 

^ j ~ 1,2,...  ,71. 


(2.3) 

(2.4) 

(2.5) 

(2.6) 


From  this  point  on,  whenever  we  write  (xij)  E X„,  it  will  be  implied  that  the  Xij 
satisfy  the  assignment  constraints  (2.4),  (2.5)  and  (2.6).  Many  authors  have  pro- 
posed methods  for  linearizing  the  quadratic  form  of  the  objective  function  in  2.3 
by  introducing  additional  variables  (see  §2.6).  The  QAP  can  be  equivalently  for- 
mulated in  a more  compact  form  if  we  define  an  inner  product.  More  specifically. 
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let  the  inner  product  between  matrices  A,  B ^ be  defined  as 

m n 

{A,  B)  :=  ^ ^ ^ ^ ciijbij. 

i=i  j=i 

Given  some  A G a permutation  p G Il;v  and  the  associated  permutation 

matrix  X G X„,  then 


AX^  : 

XA  : 


permutes  the  columns  of  A 
permutes  the  rows  of  A 


=>  XAX'^  = 


nxri' 


We  can  then  formulate  the  QAP  alternatively  as 


min  {F,  XDX^)  + {B,  X)  (2.8) 

s.t.  Xl  G ^^71* 


2.2.2  Law^ler’s  Formulation 


Lawler  [123]  introduced  a general  formulation  for  the  QAP  that  follows  nat- 
urally from  the  linear  assignment  problem.  In  the  LAP  we  are  given  cost 
coefficients  and  the  problem  is  to 


min 

s.t 


n n 


C-ij  X{j 


i=l  j=l 

{xij)  G X 


n 


It  is  well  known  that  the  LAP  can  be  solved  efficiently  in  O(n^)  time  [6].  The 
QAP  can  be  formulated  as  an  extension  of  the  LAP  with  cost  coefficients 


min 

s.t 


n n n n 


^ijklXijXj^l 


i=l  j=l  k=l  l=l 
(X^j)  G 


(2.10) 
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The  above  objective  function  reduces  to  the  one  in  formulation  (2.3)  if  we  set 
Cijki  = fikdji  iox  i ^ k and  j ^ / and  Cijij  = fudjj  + bij  to  include  the  linear  cost.  In 
the  context  of  facility  location,  Cijki  corresponds  to  the  cost  of  interaction  between 
the  facility  i at  location  j,  and  the  facility  k at  location  1.  Note  that  fudjj  = 0, 
and  Cijij  reduces  only  to  bij,  which  is  the  cost  of  assigning  facility  i to  location  j. 
Separating  quadratic  and  linear  terms  in  the  objective  function  we  obtain 

n n n n n n 

min  ^ V V (2-H) 

i=i  j=i  k=l  1=1  i=l  j=l 

S.t  ^ 'X.ji, 

As  we  shall  see  in  later  sections,  this  formulation  for  the  QAP  has  been  used  to 
obtain  lower  bounds,  and  for  the  study  of  the  QAP  polytope.  An  instance  of  the 
above  general  QAP  will  be  denoted  by  QAP{C)  and  the  objective  function  value 
for  a given  X G X„  by  Z{C,X). 

2.2.3  Concave  Quadratic  Formulation 

In  the  objective  function  of  (2.3),  let  the  products  fijdki  be  the  entries  of 
a matrix  S'„2x„2.  Now  let  Q :=  S — al,  where  I is  the  {iPx'n?)  unit  ma- 
trix, a > 1151100  s-nd  ||5||oo  is  the  row  norm  of  matrix  S.  Also  let  x = 
{xii,xi2,  ■ ■ . ,Xin,X2i,-.-  ,Xnn)'^  = (^^i,---  , Xnn)^ > and  Q to  be  Symmetric.  We 
then  have  the  quadratic  form  x^Qx,  where: 


n^  — 1 

x^Qx  = Qiixl  + 2 ^ ^ qijXiXj 

i=l  i=l  ji=i+l 

n?  n^  — 1 

i=l  j=iy/i)  i=l  j=i+l 
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n‘^  — 1 

2=1  j = l 2 = 1 jz=i-\-\ 


n‘ 


n‘ 


— *y  ^(  ^ 'y  y ^ij) 


x: 


i=l 


i=i 


Thus  we  can  conclude  that  x^Qx  < 0 for  any  x ^ 0 which  implies  that  Q is  negative 
definite  and  therefore  we  have  a quadratic  concave  minimization  problem.  More 
specifically  we  have  the  QAP  formulated  as 


min 

s.t. 


(2.12) 


Bazaraa  and  Sherali  [20]  introduced  the  above  formulation,  and  used  it  for  the 
generation  of  cutting  plane  procedures.  Although  their  exact  methods  where  not 
efficient  computationally,  heuristics  derived  from  these  procedures  produced  good 
quality  suboptimal  solutions. 


2.2.4  Trace  Formulation 

The  trace  of  a n x n matrix  B is  defined  to  be  the  sum  of  its  diagonal  elements, 
that  is, 

n 

trB  :=  yhij. 

i=l 

Consider  a QAP  instance  with  input  matrices  F,  D and  B.  Letting  D = XD^X^, 


then 
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n n n n 

tr{FD)  — fij^ji  ~ fijdp{i)p{j)^ 

2=1  j = l 2=1  J = 1 

since  dji  = dp^i)p(^j),  i,j  = 1, . . . ,n,  where  p G Hat  is  the  permutation  associated 
with  X (see  (2.7)).  Since  tr(BX^)  = QAP  can  be  formulated 

analogously  with  formulation  (2.8),  as 


min  tr{FXD^  + B)X^ 


XgX„. 


(2.13) 


The  trace  formulation  of  the  QAP  first  appeared  in  Edwards  [63,  64],  while  it  was 
used  by  Finke  et  al.  [71]  for  introducing  the  eigenvalue  lower  bounding  techniques 
for  symmetric  QAPs  (see  §2.9.1).  Given  any  two  matrices  A,  .B  G IR”^",  recall  the 
well  known  properties  {ABY  = BX and  trA  = trA^ . For  F = F^  we  can  then 
write  the  quadratic  term  in  (2.13)  as 

trFXD^X'^  = trFXDX^, 


where  D is  not  necessarily  symmetric.  Therefore,  given  a QAP  instance  with  only 
one  of  the  matrices  symmetric  (say  F),  we  can  transform  it  into  a QAP  instance 
with  both  matrices  symmetric.  This  is  done  by  the  introduction  of  a symmetric 
matrix  E = \{D  + D^),  since 

trFXE^X^  = ]-tr{EXD'^X^  + FXDX'^)  = trEXD'^X'^. 


2.2.5  Kronecker  Product 

Let  A G and  B G Then  the  Kronecker  Product  of  the  matrices  A 


and  B is  defined  as 
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diiB  CL12B 


Oj\nB 


\ 


A®B  \ 


(^mlB  (I1J12B  • • • OtjyijiB 


J 


That  is,  the  mp  x nq  matrix  formed  from  all  possible  pairwise  element  products 

2 

of  A and  B.  If  we  let  vecyX)  G IR"  be  the  vector  formed  by  the  columns  of  a 
permutation  matrix  X,  the  QAP  can  be  formulated  as 


min  vec{X)^ {F  <S>  D)vec{X)  + vec{B)^vec{X)  (2-14) 

s.t.  X G X.JI. 


Although  operations  using  the  kronecker  product  have  been  studied  in  detail  [88], 
the  above  formulation  is  rarely  used  for  studying  the  QAP.  Using  the  kronecker 
product,  Lawler  [123]  provided  an  alternative  formulation  for  the  QAP  as  an  ri^xiF 
LAP.  An  x matrix  C is  constructed  from  the  rA  costs  Cijku  such  that  the 
{ijkiy^  element  corresponds  to  the  {{i  — l)n  + k,  {j  — l)n  + element  of  C.  The 
QAP  then  is  equivalent  to  an  LAP  of  dimension  rA  with  C as  the  cost  matrix,  and 
with  the  additional  constraint  that  the  x rA  permutation  matrix  which  defines 
a feasible  solution,  must  be  the  kronecker  product  of  two  permutation  matrices  of 
dimension  n.  In  other  words  the  QAP  is  equivalent  to 

min  {C,  Y) 

s.t.  Y = X®X,  (2.15) 

X G X„. 

The  resulting  LAP  however  cannot  be  solved  efficiently  (i.e.,  in  0{rA)  time)  because 
y,  although  it  is  an  rA  x rA  permutation  matrix,  is  constrained  to  have  a special 
structure  as  it  will  be  demonstrated  in  §2.6.4. 
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2.3  Computational  Complexity 


The  results  described  in  this  section  bring  evidence  to  the  fact  that  the  QAP  is 
a “very  hard”  problem  from  the  theoretical  point  of  view.  Not  only  that  the  QAP 
cannot  be  solved  efficiently  but  it  even  cannot  be  approximated  efficiently  within 
some  constant  approximation  ratio.  Furthermore,  finding  local  optima  is  not  a 
trivial  task  even  for  simply  structured  neighborhoods  like  the  2-opt  neighborhood. 

Two  early  results  obtained  by  Sahni  and  Gonzalez  [174]  in  1976  settled  the 
complexity  of  solving  and  approximating  the  QAP.  It  was  shown  that  the  QAP  is 
NP-hard  and  that  even  finding  an  e-approximate  solution  for  the  QAP  is  a hard 
problem,  in  the  sense  that  the  existence  of  a polynomial  e-approximation  algorithm 
implies  P=NP.  In  the  following,  let  Z(F,  D,p)  denote  the  objective  function  value 
of  a solution  p for  a QAP  with  flow  matrix  F and  distance  matrix  D. 


Definition  2.1  Given  a real  number  c > 0,  an  algorithm  T for  the  QAP  is  said 
to  be  an  e-approximation  algorithm  if 


Z{F,  D,  7Tt)  - Z{F,  D,  TTopt) 

'^opi) 


(2.16) 


holds  for  every  instance  QAP{F,  D),  where  ttx  is  the  solution  of  QAP{F,  D)  com- 
puted by  algorithm  T and  is  an  optimal  solution  ofQAP{F,  D).  The  solution 
of  QAP {F,  D)  produced  by  an  e- approximation  algorithm  is  called  an  e-approximate 
solution. 


Theorem  2.1  (Sahni  and  Gonzalez  [174],  1976) 

The  quadratic  assignment  problem  is  strongly  NP-hard.  For  an  arbitrary  e > 0, 
the  existence  of  a polynomial  time  e- approximation  algorithm  for  the  QAP  implies 
V = NfV. 
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The  proof  is  done  by  a reduction  from  the  Hamiltonian  cycle  problem:  Given  a 
graph  G,  does  G contain  a cycle  which  visits  each  vertex  exactly  once  [78]? 

Queyranne  [165]  derives  an  even  stronger  result  which  further  confirms  the 
widely  spread  belief  on  the  inherent  difficulty  of  the  QAP  in  comparison  with  other 
difficult  combinatorial  optimization  problems.  It  it  well  known  and  very  easy  to 
see  that  the  traveling  salesman  problem  (TSP)  is  a special  case  of  the  QAP.  The 
TSP  on  n cities  can  be  formulated  as  a QAP{F,  D)  where  F is  the  distance  matrix 
of  the  TSP  instance  and  D is  the  adjacency  matrix  of  a Hamiltonian  cycle  on  n 
vertices.  In  the  case  that  the  distance  matrix  is  symmetric  and  satisfies  the  triangle 
inequality,  the  TSP  is  approximable  in  polynomial  time  within  3/2  as  shown  by 
Christofides  [49].  Queyranne  [165]  showed  that,  unless  P=NP,  QAP{A,  B)  is  not 
approximable  in  polynomial  time  within  some  finite  approximation  ratio,  even  if 
A is  the  distance  matrix  of  some  set  of  points  on  a line  and  H is  a symmetric  block 
diagonal  matrix. 

A more  recent  result  of  Arora,  Frieze  and  Kaplan  [10]  answers  partially  one  of 
the  open  questions  stated  by  Queyranne  [165].  What  happens  if  matrix  A is  the 
distance  matrix  of  n points  which  are  regularly  spaced  on  a line,  that  is,  points  with 
abscissae  given  by  Xp  = p,  p = 1, . . . ,n?  This  special  case  of  the  QAP  is  termed 
linear  arrangement  problem  and  is  a well  studied  NP-hard  problem.  In  the  linear 
arrangement  problem  the  matrix  B is  not  restricted  to  have  the  block  diagonal 
structure  mentioned  above,  but  is  simply  a symmetric  0-1  matrix.  Arora  et  al. 
give  a polynomial  time  approximation  scheme  (PTAS)  for  the  linear  arrangement 
problem  in  the  case  that  the  0-1  matrix  B is  dense,  that  is,  the  number  of  1 entries 
in  B is  in  f2(n^),  where  n is  the  size  of  the  problem.  They  show  that  for  each 
e > 0 there  exists  an  c-approximation  algorithm  for  the  dense  linear  arrangement 
problem  with  time  complexity  depending  polynomially  on  n and  exponentially  on 
1/e,  hence  polynomial  for  each  fixed  e > 0. 
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Recently  it  has  been  shown  that  even  finding  a locally  optimal  solution  of  the 
QAP  can  be  prohibitively  hard,  that  is,  even  local  search  is  hard  in  the  case  of  the 
QAP.  Below  we  formalize  this  idea  to  some  extent.  Assume  that  an  optimization 
problem  P is  given  by  specifying  a ground  set  a set  P C 2^  oi  feasible  solutions 
and  a cost  function  c:  £ — > IR.  This  cost  function  c implies  an  objective  function 
/:  ^ IR  defined  by  f{S)  = S e P.  The  goal  is  to  find  a 

feasible  solution  which  minimizes  the  objective  function.  For  every  feasible  solution 
S'  G .F  let  a neighborhood  M{S)  C P of  S he  given.  This  neighborhood  consists  of 
feasible  solutions  which  are  somehow  “close”  to  S.  Now,  instead  of  looking  for  a 
globally  optimal  solution  S*  e P of  the  problem  P,  that  is, 

f{S*)  = 

we  look  for  a locally  optimal  solution  or  a local  minimum  of  P,  that  is  an  5 G .F 
such  that 

= min 
seAf(S) 

An  algorithm  which  produces  a locally  optimal  solution  is  frequently  called  a local 
search  algorithm.  Some  local  search  algorithms  for  the  QAP  are  described  in  §2.10. 

Let  us  consider  the  intriguing  question  “Is  it  easy  to  find  a locally  optimal 
solution  for  the  QAP?”  Clearly  the  answer  depends  on  the  involved  neighborhood 
structure.  If  the  neighborhoods  A/"(5)  are  replaced  by  new  neighborhoods  J\f'{S), 
one  would  generally  expect  changes  in  the  local  optimality  status  of  a solution. 
The  theoretical  basis  for  facing  this  kind  of  problems  was  introduced  by  Johnson, 
Papadimitriou  and  Yannakakis  [101].  They  define  the  so-called  polynomial-time 
local  search  problems,  shortly  PLS  problems.  A pair  (P,A/*),  where  P is  a (combi- 
natorial) optimization  problem  P and  M is  an  associated  neighborhood  structure. 
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defines  a local  search  problem  which  consists  of  finding  a locally  optimal  solution 
of  P with  respect  to  the  neighborhood  structure  J\f.  Without  going  into  technical 
details  a PLS  problem  is  a local  search  problem  for  which  local  optimality  can  be 
checked  in  polynomial  time.  In  analogy  with  decision  problems,  there  exist  com- 
plete problems  in  the  class  of  PLS  problems.  The  PLS-complete  problems,  are-in 
the  usual  complexity  sense-the  most  difficult  among  the  PLS  problems. 

Murthy,  Pardalos  and  Li  [145]  introduce  a neighborhood  structure  for  the  QAP 
which  is  similar  to  the  neighborhood  structure  proposed  by  Kernighan  and  Lin  [113] 
for  the  graph  partitioning  problem.  For  this  reason  we  will  call  it  a K-L  type 
neighborhood  structure  for  the  QAP.  Murthy  et  al.  show  that  the  corresponding 
local  search  problem  is  PLS-complete. 

Consider  a permutation  po  ^ Hyv-  A swap  of  po  is  a permutation  p G Hyv 
obtained  from  po  by  applying  a transposition  {i,j)  to  it,  p = Po  o (hj)-  A trans- 
position (i,j)  is  defined  as  a permutation  which  maps  i to  j,  j to  i,  and  k to  k for 
all  k ^ {i,  j}.  In  the  facility  location  context  a swap  is  obtained  by  interchanging 
the  facilities  assigned  to  two  locations  i and  j.  A greedy  swap  of  permutation  po  is 
a swap  Pi  which  minimizes  the  difference  Z{F,D,p)  — Z{F,D,po)  over  all  swaps 
p of  Po.  Let  po,pi, . . . ,pi  be  a set  of  permutations  in  I1;V)  each  of  them  being  a 
greedy  swap  of  the  preceding  one.  Such  a sequence  is  called  monotone  if  for  each 
pair  of  permutations  Pk  (pt)  in  the  sequence,  {ik,jk}  C {it,jt}  = 0>  where  pk  (pt)  is 
obtained  by  applying  transposition  {ik,jk)  {{it,jt))  to  the  preceding  permutation 
in  the  sequence.  The  neighborhood  of  po  consists  of  all  permutations  which  occur 
in  the  (unique)  maximal  monotone  sequence  of  greedy  swaps  starting  with  permu- 
tation Po.  Let  us  denote  this  neighborhood  structure  for  the  QAP  by  Mk-l-  It  is 
not  difficult  to  see  that,  given  a QAP{F,  D)  of  size  n and  a permutation  p G II//, 
the  cardinality  of  AfK-iip)  does  not  exceed  [n/2j  -I- 1. 
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It  is  easily  seen  that  the  local  search  problem  {QAP,Mk-l)  is  a PLS  problem. 
Pardalos,  Rendl,  and  Wolkowicz  [157]  have  shown  that  a PLS-complete  problem, 
namely  the  graph  partitioning  problem  with  the  neighborhood  structure  defined 
by  Kernighan  and  Lin  [113]  is  PLS-reducible  to  {QAP,Mk-l)-  This  implies  the 
following  result. 

Theorem  2.2  (Pardalos,  Rendl  and  Wolkowicz  [157],  1994) 

The  local  search  problem  {QAP^Mk-l),  where  Mk-l  is  the  Kernighan-Lin  type 
neighborhood  structure  for  the  QAP,  is  PLS-complete. 

The  PLS-completeness  of  {QAP^Mk-l)  implies  that,  in  the  worst  case,  a gen- 
eral local  search  algorithm  as  described  above  involving  the  Kernighan-Lin  type 
neighborhood  finds  a local  minimum  only  after  a time  which  is  exponential  on  the 
problem  size.  Numerical  results,  however,  show  that  such  local  search  algorithms 
perform  quite  well  when  applied  to  QAP  test  instances,  as  reported  in  Murphy  et 
al.  [145]. 

Another  simple  and  frequently  used  neighborhood  structure  in  n^v  is  the  so- 
called  pair-exchange  (or  2-opt)  neighborhood  A/2.  The  pair-exchange  neighborhood 
of  a permutation  po  ^ Ryv  consists  of  all  permutations  p G fl^v  obtained  from  po 
by  applying  some  transposition  (i,j)  to  it.  Thus,  M^ip)  = {p°  {i,j)  \ 1 < i,j  < 
n,  i^j}. 

It  can  also  be  shown  that  (QAP,  A/2)  is  PLS-complete.  Schaffer  and  Yan- 
nakakis  [175]  have  proven  that  the  graph  partitioning  problem  with  a neighbor- 
hood structure  analogous  to  A/2  is  PLS-complete.  A similar  PLS-reduction  as  in 
Pardalos  et  al.  [157]  implies  that  the  local  search  problem  (QAP,  A/2),  where  A/2  is 
the  pair-exchange  neighborhood,  is  PLS-complete.  This  implies  that  the  time  com- 
plexity of  a general  local  search  algorithm  for  the  QAP  involving  the  pair-exchange 
neighborhood  is  also  exponential  in  the  worst  case. 
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Finally,  let  us  mention  that  no  local  criteria  are  known  for  deciding  how  good  a 
locally  optimal  solution  is  as  compared  to  a global  one.  From  the  complexity  point 
of  view,  deciding  whether  a given  local  optimum  is  a globally  optimal  solution  to  a 
given  instance  of  the  QAP  is  a hard  problem,  see  Papadimitriou  and  Wolfe  [152]. 

2.4  Asymptotic  Behavior 

The  QAP  shows  an  interesting  asymptotic  behavior:  under  certain  probabilistic 
conditions  on  the  coefficient  matrices  the  QAP,  the  ratio  between  its  “best”  and 
“worst”  values  of  the  objective  function  approaches  1,  as  the  size  of  the  problem 
approaches  infinity.  This  asymptotic  behavior  suggests  that  the  relative  error  of 
every  heuristic  method  vanishes  as  the  size  of  the  problem  tends  to  infinity,  that 
is,  every  heuristic  finds  almost  always  an  almost  optimal  solution  when  applied  to 
QAP  instances  which  are  large  enough.  In  other  words  the  QAP  becomes  in  some 
sense  trivial  as  the  size  of  the  problem  tends  to  infinity.  Burkard  and  Fincke  [34] 
identify  a common  combinatorial  property  of  a number  of  problems  which,  under 
natural  probabilistic  conditions  on  the  problem  data,  behave  as  described  above. 
This  property  seems  to  be  also  the  key  for  the  specific  asymptotic  behavior  of  the 
QAP. 

In  an  early  work  Burkard  and  Fincke  [33]  investigate  the  relative  difference 
between  the  worst  and  the  best  value  of  the  objective  function  for  Koopmans- 
Beckmann  QAPs.  They  first  consider  the  case  where  the  coefficient  matrix  D 
is  the  matrix  of  pairwise  distances  of  points  chosen  independently  and  uniformly 
from  the  unit  square  in  the  plane.  Then  the  general  case  where  entries  of  the  flow 
and  distance  matrices  F and  D are  independent  random  variables  taken  from  a 
uniform  distribution  on  [0, 1]  is  considered.  In  both  cases  it  is  shown  that  the 
relative  difference  mentioned  above  approaches  0 with  probability  tending  to  1 as 
the  size  of  the  problem  tends  to  infinity. 
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Later  Burkard  and  Fincke  [34]  consider  the  ratio  between  the  objective  function 
values  corresponding  to  an  optimal  (or  best)  and  a worst  solution  of  a generic 
combinatorial  optimization  problem  described  below. 

Consider  a sequence  P„,  n G IN,  of  combinatorial  optimization  (minimization) 
problems  with  sum  objective  function  as  described  in  §2.3.  Let  and  J-n  be  the 
ground  set  and  the  set  of  feasible  solutions  of  problem  Pn,  respectively.  Moreover, 
let  Cn'.  Sn  — t IR’*’  and  / : P -4  IR"^  be  the  nonnegative  cost  function  and  the 
objective  function  for  problem  respectively.  For  n G IN,  an  optimal  solution  X^pt 
minimizes  the  objective  function,  whereas  a worst  solution  G maximizes 
the  objective  function  and  is  defined  as  follows: 

) = XI  ^n{x)  = max f{X)  = max  X Cn(x 

It  is  shown  in  Burkard  et  al.  [34]  that  the  behavior  of  the  ratio  f(Xppt)/ f(X„„) 
is  strongly  related  to  the  ratio  ln|jF„|/|X„|  between  the  cardinality  of  the  set  of 
feasible  solutions  and  the  cardinality  of  an  arbitrary  feasible  solution  Xn,  under 
the  assumption  that  all  feasible  solutions  have  the  same  cardinality. 

Theorem  2.3  (Burkard  and  Fincke  [34],  1985) 

Let  Pn  be  a sequence  of  combinatorial  minimization  problems  with  sum  objective 
function  as  described  above.  Assume  that  the  following  conditions  are  fulfilled: 

(BFl)  For  all  X G Tn,  j^j  = where  is  some  feasible  solution  in  Xn- 

(BF2)  The  costs  Cn{x)  , X e X,  X e Xn,  n e IN,  are  random  variables  identically 
distributed  on  [0,1].  The  expected  value  E = E(cn(x))  and  the  variance 
cr^  = a‘^[cn{x))  > 0 of  the  common  distribution  are  finite.  Moreover,  for  all 
X G Xn,  n G IN,  the  variables  Cn{x),  x £ X , are  independently  distributed. 
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(BF3)  \J^n\  and  tend  to  infinity  as  n tends  to  infinity  and  moreover, 


lim  +oo 


n->oo 


where  Aq  is  defined  by  Aq  :=  (eoO"/(eo  + 2o^)Y  and  cq  fulfills 


jE*  4-  6 

0 < eo  < and  0 < < 1 + e , 

E - €o 


(2.17) 


for  a given  e > 0. 


Then,  as  n — > oo 


min  X)  Cn{x) 


> l-2|J^„|exp(-|X(")|Ao)  ->1. 


The  combinatorial  condition  represented  by  the  limit  in  (BF3)  says  that  the 
cardinality  of  the  feasible  solutions  is  large  enough  with  respect  to  the  cardinality 
of  the  set  of  feasible  solutions.  Namely,  the  result  of  the  theorem  is  true  if  the 
following  equality  holds; 


lim 


The  other  conditions  of  Theorem  2.3  are  natural  probabilistic  requirements  on 
the  coefficients  of  the  problem.  Theorem  2.3  states  that  for  each  e > 0,  the  ratio 
between  the  best  and  the  worst  value  of  the  objective  function  lies  on  (1  — e,  1 + e), 
with  probability  tending  to  1 , as  the  “size”  of  the  problem  approaches  infinity.  Thus, 
we  have  convergence  with  probability.  Under  one  additional  natural  (combinatorial) 
assumption  (condition  (S3)  of  the  theorem  below),  Szpankowski  strengthens  this 
result  and  improves  the  range  of  the  convergence  to  almost  surely.  In  the  almost 
sure  convergence  the  probability  that  the  above  mentioned  ratio  tends  to  1 is  equal 
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to  1.  (Detailed  explanations  on  the  probabilistic  notions  used  can  be  found  in  every 
text  book  on  probability  theory.) 

Theorem  2.4  (Szpankowski  [181],  1995) 

Let  Pn  be  a sequence  of  combinatorial  minimization  problems  with  sum  objective 
function  as  above.  Assume  that  the  following  conditions  are  fulfilled: 

(51)  For  all  X G |X|  = where  is  some  feasible  solution  in  Tn- 

(52)  The  costs  Cn{x) , x E X , X £ Tn,  n G IN,  are  random  variables  identically 
and  independently  distributed  on  [0,1].  The  expected  value  E = E{cn  i^)), 
the  variance,  and  the  third  moment  of  the  common  distribution  are  finite. 

(53)  The  worst  values  of  the  objective  function,  max  ^ Cn(x),  form  a nonde- 
creasing  sequence  for  increasing  n. 

(54)  \Tn\  and  tend  to  infinity  as  n tends  to  infinity  and  moreover.  In  \Tn\  = 

o(|X(”)|), 

Therij  the  following  equalities  hold  almost  surely: 


min 

XeTn 


53c„W  = |Jf<-»|£ 


max  Y,  + o(|A'<">|) 

A t-/* n 

xEX 

Theorems  2.3  and  2.4  can  be  applied  to  the  QAP.  The  reason  is  that  the  QAP  fulfills 
the  combinatorial  condition  (S4)  in  Theorem  2.4  (and  therefore,  also  condition 
(BF3)  in  Theorem  2.3).  Thus,  we  immediately  get  the  following  corollary; 

Corollary  2.1  Consider  a sequence  of  problems  QAP{A^'^\  B("^)  for  n G IN,  with 
nx  n coefficient  matrices  = (alj)  and  B — Assume  that  and 
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in') 

b\j  , n e 1 < i,j  < n,  are  independently  distributed  random  variables  on 
[0,  M],  where  M is  a positive  constant.  Moreover,  assume  that  entries  n G IN, 
1 ^ h3  ^ n,  have  the  same  distribution,  and  entries  b\^\  n E 1 < i,j  < n, 
have  also  the  same  distribution  (which  does  not  necessarily  coincide  with  that  of 
a\j).  Furthermore,  assume  that  these  variables  have  finite  expected  values,  vari- 
ances and  third  moments. 

Let  pf^t  and  p^fi2-  denote  an  optimal  and  a worst  solution  of  QAP{A^'^\  re- 

spectively, that  is. 


min  Z (A^’^\B^'^\p) 

peUN  ^ ’ 


and 


max  Z (A^^\  B^^\p) 

peiiN  ^ ' 


Then  the  following  equality  holds  almost  surely: 


lim 

n— >oo 


The  above  result  suggests  that  the  value  of  the  objective  function  of  the  problem 
QAP{A^'^\  (corresponding  to  an  arbitrary  feasible  solution)  gets  somehow 
close  to  its  expected  value  n^E{A)E{B),  as  the  size  of  the  problem  increases, 
where  E{A)  and  E{B)  are  the  expected  values  of  and  b\'j\  n G IN,  1 < 
fj  < respectively.  Frenk,  Houweninge,  and  Rinnooy  Kan  [73]  and  Rhee  [170, 
171]  provide  different  analytical  evaluations  for  this  “getting  close”,  by  imposing 
different  probabilistic  conditions  on  the  data.  The  following  theorem  states  two 
important  results. 
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Theorem  2.5  (Frenk  et  al.  [73],  1986,  Rhee  [171],  1991) 

Consider  the  sequence  of  QAP{A^'^\  n € IN,  as  in  Corollary  2.1.  Assume 

that  the  following  conditions  are  fulfilled: 


(Cl)  aS"',  6j">,  n G IN,  1 < < n,  are  random  variables  independently  dis- 

tributed on  [0,M]. 


(C2)  ajf,  n G IN,  1 < i,  j < n,  have  the  same  distribution  on  [0,  M].  b^A\  n G IN, 
1 ^ ^ have  also  the  same  distribution  on  [0,  M]. 


Let  E{A),  E{B)  be  the  expected  values  of  the  variables  and  bf^\  respectively. 
Then,  there  exists  a constant  K\  (which  does  not  depend  on  n),  such  that  the 
following  inequality  holds  almost  surely,  forp  G 11;^)  G IN 


lim  sup 

n-Aoo 


\/Iogn 


n‘^E{A)E{B)  ~ 


Moreover,  let  Y be  a random  variable  defined  by 


Y = Z - n‘^E{A)E{B) , 

where  p^(f]  is  an  optimal  solution  of  QAP{A^'^\  B^'^^). 

Then  there  exists  another  constant  K^,  also  independent  of  the  size  of  the  prob- 
lem, such  that 


■^n^^^{\ognY^^  < E{Y)  < K^n^^^ ifognY^^ 
iv2 


-f 


P{\Y-E{Y)\  >t}<  2 exp 


4n^A\\UB\\l 
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for  each  t > 0,  where  E(Y)  denotes  the  expected  value  of  variable  Y and  ||>1|| 


OO 


f||B||oo^  is  the  so-called  row  sum  norm  of  matrix  A (B)  defined  by  ||^||oo  = 


a 


jji- 


These  results  on  the  asymptotic  behavior  of  the  QAP  have  been  exploited 
by  Dyer,  Frieze,  and  McDiarmid  [60]  to  analyze  the  performance  of  branch  and 
bound  algorithms  for  QAPs  with  coefficients  generated  randomly  as  described 
above.  Dyer  et  al.  have  shown  that  for  such  QAPs  the  optimal  value  of  the  contin- 
uous relaxation  of  Frieze  and  Yadegar’s  linearization  (2.29)-(2.36)  is  in  0(n)  with 
probability  tending  to  1 as  the  size  n of  the  QAP  tends  to  infinity.  Hence  the  gap 
between  the  optimal  value  of  this  continuous  relaxation  and  the  optimal  value  of 
the  QAP  grows  like  0{n)  with  probability  tending  to  1 as  n tends  to  infinity.  This 
result  leads  to  the  following  theorem. 

Theorem  2.6  (Dyer,  Frieze,  and  McDiarmid  [60],  1986) 

Consider  any  branch  and  bound  algorithm  for  solving  a QAP  with  randomly  gen- 
erated coefficients  as  in  Corollary  2.1,  that  uses  single  assignment  branching  and 
employs  a bound  obtained  by  solving  the  continuous  relaxation  of  the  linearization 
(2.29)-(2.36).  The  number  of  branched  nodes  explored  is  at  least 
probability  tending  to  1 as  the  size  n of  the  QAP  tends  to  infinity. 

2.5  QAP  Polytopes 


A polyhedral  description  of  the  QAP  and  of  some  of  his  relatives  have  been 
recently  investigated  by  Barvinok  [17],  Padberg  and  Rijal  [149,  172],  as  well  as  by 
Junger  and  Kaibel  [104,  105]  and  Kaibel  [106].  Although  in  an  early  stage  yet, 
the  existing  polyhedral  theory  around  the  QAP  counts  already  a number  of  results 
concerning  basic  features  like  dimensions,  affine  hulls,  and  valid  and  facet  defining 
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inequalities  in  the  case  of  the  general  QAP  polytope  and  of  the  symmetric  QAP 
polytope. 

The  linearization  of  Frieze  and  Yadegar  introduced  in  the  previous  section  can 
be  used  as  a starting  point  for  the  definition  of  the  QAP  polytope.  The  QAP 
polytope  is  defined  as  a convex  hull  of  all  0-1  vectors  (xij,yijkt),  1 < < n, 

which  are  feasible  solutions  of  the  MILP  formulation  of  Frieze  and  Yadegar  [75]. 

Another  possibility  to  introduce  the  QAP  polytope  is  the  formulation  of  the 
QAP  as  a graph  problem  as  proposed  by  Jiinger  and  Kaibel  [104].  This  formulation 
provides  some  additional  insight  in  the  problem  and  allows  an  easier  use  of  some 
technical  tools,  for  example,  projections  and  affine  transformations.  The  latter  lead 
to  a better  understanding  of  the  relationship  between  the  general  QAP  polytope 
and  related  polytopes,  for  example,  the  symmetric  QAP  polytope,  or  well  studied 
polytopes  of  other  combinatorial  optimization  problems  like  the  traveling  salesman 
polytope  or  the  cut  polytope  (see  [106]). 

For  each  n € IN  consider  a graph  = (14,  with  vertex  set  14  = 
{(bi)  I 1 < i,j  < n}  and  edge  set  = {{{i , j) , {k , 1)) : i k,j  7^  /}.  Clearly, 
the  maximal  cliques  in  G„  have  cardinality  n and  correspond  to  the  permutation 
matrices.  Given  an  instance  of  the  Lawler  QAP  with  coefficients  Cijki  and  linear 
term  coefficients  bij,  we  introduce  as  vertex  weights  and  Cijki  as  weights  of  the 
edge  {{i,j),  {k,  /)).  Solving  the  above  QAP  instance  is  equivalent  to  finding  a max- 
imal clique  with  minimum  total  vertex-  and  edge-weight.  For  each  clique  C in 
Gn  with  n vertices  we  denote  its  incidence  vector  by  (x^,  y^),  where  e IR”  , 


/ 


/ 


1 if(bj)eC, 


1 if  (z,i),(A:,/)  G C, 


0 otherwise 


yijkl  ^ 


0 otherwise 
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The  QAP  polytope  denoted  by  QAP^  is  then  given  by 

QAPn  :=  conv{{pr ,y^)  | C is  a clique  in  with  n vertices}. 


It  turns  out  that  the  traveling  salesman  polytope  and  the  linear  ordering  polytope 
are  projections  of  QAPn,  and  that  QAPn  is  a face  of  the  Boolean  quadric  polytope, 
see  [106]. 

Barvinok  [17],  Padberg  and  Rijal  [149],  and  Jiinger  and  Kaibel  [104]  have  inde- 
pendently computed  the  dimension  of  QAPn,  and  have  shown  that  the  inequalities 
Vijki  > 0,  i ^ A;,  j 7^  /,  are  facet  defining.  (These  are  usually  called  trivial  facets 
of  QAPn-  Moreover,  Padberg  and  Rijal  [149],  and  Junger  and  Kaibel  [104]  have 
independently  shown  that  the  affine  hull  of  QAPn  is  described  by  the  following 
equations  which  are  linearly  independent: 

n 

= 1,  It  < j < n,  (2.18) 

i=l 

n 

^ ^ Xjj  — 1,  1 ^ ^ ^ 

i=i 


fc— 1 n 

Xkl  T ^ ^ yijkl  T / ^ Vklij  0 

i=l  i=fc+l 


1 < j ^ I < n,l  < k < n — 1, 
or  l<l<j<n,k  = n 


(2.20) 


j — l n 

Xij  -f"  ^ ] yijkl  T ^ ^ yijkl  0 
J=1  l=j+l 


i < k < n — 1 or 
1 ^ j < n — l,i  = n — 2, 


(2.21) 


k = n — 1 
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l</<n  — l,l<i<n  — 3, 

(2.22) 

i < k < n — 1 

Summarizing  we  get  the  following  theorem: 

Theorem  2.7  (Barvinok  [17],  1992,  Jiinger  and  Kaibel  [104],  1996,  Padberg  and 
Rijal  [149],  1996) 

(i)  The  affine  hull  of  the  QAP  polytope  QAPn  is  given  by  the  linear  equations 
(2.18)-(2.22).  These  equations  are  linearly  independent  and  the  rank  of  the 
system  is  2n(n  — 1)^  — (n  — l)(n  — 2). 

(a)  The  dimension  of  QAP^  is  equal  to  1 + (n  — 1)^  4-  n(n  — l)(n  — 2)(n  — 3)/2. 

(Hi)  The  inequalities  yijki  > 0,  i < k,  j ^ I,  define  facets  of  QAPn- 

Padberg  and  Rijal  [149]  identified  additionally  two  classes  of  valid  inequalities 
for  QAPn,  the  clique  inequalities  and  the  cut  inequalities,  where  the  terminology  is 
related  to  the  graph  The  authors  identify  some  conditions  under  which  the  cut 
inequalities  are  not  facet  defining.  It  is  an  open  problem,  however,  to  identify  facet 
defining  inequalities  within  these  classes.  A larger  class  of  valid  inequalities,  the 
so-called  box  inequalities  have  been  described  by  Kaibel  [106].  Those  inequalities 
are  obtained  by  exploiting  the  relationship  between  the  Boolean  quadric  polytope 
and  the  QAP  polytope.  A nice  feature  of  the  box  inequalities  is  that  it  can  be 
decided  efficiently  whether  they  are  facet  defining  or  not,  and  in  the  latter  case 
some  facet  defining  inequality  which  dominates  the  corresponding  box  inequality 
can  be  derived. 

Similar  results  have  been  obtained  for  the  symmetric  QAP  polytope,  SQAPn, 
arising  in  the  case  that  at  least  one  of  the  coefficient  matrices  of  the  given  QAP 
(matrices  F,  D in  (2.1))  is  symmetric.  The  definition  of  SQAPn  is  given  by  means 
of  a hypergraph  Hn  = {Vn,  Fn),  where  14  is  the  same  set  of  vertices  as  in  graph 


1—1  n 

—Xkl  + 22i  y^kl  + X]  Vijkl  = 0 
j=l  3=1+1 
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and  Fn  is  the  set  of  hyperedges  {k,  1),  (i,  1),  {k,j)}  for  all  i ^ k,  j ^ 1.  A set 

C C 14  is  called  a clique  in  Hn  if  it  is  a clique  in  Again,  the  incidence  vector 
{x^,  y^)  of  a clique  C is  introduced  by 


1 if(i,i)€C 


0 otherwise 


iii<k,l  ^ j,  {i,j),{k,l)  G C 
otherwise 


Here,  G IR"*  and  G IR  ^ . The  polytope  SQAPn  is  then  defined  as 


,c 


SQAPn  :=  conv{{x^ ,y^)  | (7  is  a clique  in  G„with  n vertices} 


Padberg  and  Rijal  [149]  as  well  as  Jiinger  and  Kaibel  [105]  showed  that  the 
following  system  of  equations  (2.23)-(2.26)  given  offers  a minimal  linear  description 


of  the  affine  hull  of  SQAPn- 


n 


i=l 


j-1  n 

^ ^ Vilkj  “1“  ^ ^ Vijkl 

l=l 


k—1  n 

^kj  ^kl  “1“  ^ ^ Vijkl  “t“  ^ ^ Vkjil 

i=l  i=k+l 


1 < i < n. 

(2.23) 

1 < y < n — 1, 

(2.24) 

0, 

1 < i < k < n, 

(2.25) 

1 <j  <n, 

1 < k < n, 

0, 

1 < j < n — 3, 

(2.26) 

1 < j < f < n — 

1. 

Jiinger  and  Kaibel  [105]  proved  a conjecture  of  Padberg  and  Rijal  concerning  the 
dimension  of  SQAP^.  They  also  introduced  a class  of  facet  defining  inequalities,  so- 
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called  curtain  inequalities.  The  separation  problem  for  these  inequalities  has  been 
shown  to  be  NP-hard.  Summarizing  these  results  we  get  the  following  theorem 

Theorem  2.8  (Jiinger  and  Kaibel  [105],  1996,  Padberg  and  Rijal  [149],  1996) 

(i)  The  affine  hull  of  the  symmetric  QAP  polytope  SQAPn  is  described  by  the 
linear  equations  (2.23)-(2.26).  These  equations  are  linearly  independent  and 
the  rank  of  the  system  is  n^(n  — 2)  + 2n  — 1. 

(a)  The  dimension  of  SQAPn  is  equal  to  (n  — 1)^  + n^{n  — 3)^/4. 

(in)  The  inequalities  yijki  > 0 for  i < k,  j < I,  and  Xij  > 0 for  1 < i,j  < n, 
define  facets  of  QAPn- 

(iv)  For  each  i < k and  for  a//  J C {1,  2, . . . , n}  the  row  curtain  inequalities 


^ ^ ^ij  “I”  ^ ^ Vijkl  — 0 
jeJ  j,iej 

3<l 


are  valid  for  SQAP^.  For  each  j < I and  for  all  I C {1,  2, . . . , n}  the  column 
curtain  inequalities 


^ ^ ^ij  ”1“  ^ ^ Vijkl  ^ 0 

i,k^I 

i<k 

are  valid  for  SQAPn- 

All  curtain  inequalities  with  3 < ]/|,  |J[  < n — 3 define  facets  of  SQAPn-  The 
other  curtain  inequalities  define  faces  which  are  contained  in  trivial  facets  of 

SQAPn . 

Finally,  there  are  some  additional  results  concerning  the  affine  description  and 
the  facial  structure  of  polytopes  of  special  versions  of  sparse  QAPs,  for  example, 
sparse  Koopmans-Beckmann  QAPs,  see  Kaibel  [106].  The  idea  is  to  take  advantage 
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of  the  sparsity  for  a better  analysis  and  description  of  the  related  polytopes.  These 
investigations,  however,  are  still  in  their  infancy. 


2.6  Linearizations 

The  first  attempt  to  devise  solution  techniques  for  solving  the  QAP  had  to  do 
with  the  elimination  of  the  quadratic  term  in  the  objective  function  of  (2.10),  in 
order  to  transform  the  problem  into  a 0-1  linear  program.  The  linearization  of 
the  objective  function  is  usually  achieved  with  the  introduction  of  new  variables, 
and  new  linear  (and  binary)  constraints.  Four  such  linearizations  of  the  QAP 
will  be  presented  in  this  section.  The  first  is  due  to  Lawler  [123],  which  is  the  first 
linearization  suggested  for  the  QAP,  and  the  second  by  Kaufman  and  Broeckx  [112] 
which  is  the  smallest  with  regard  to  the  number  of  new  variables  and  constraints 
introduced.  The  third  is  a more  recent  one  that  is  due  to  Adams  and  Johnson  [3], 
which  unifies  most  of  the  previous  linearizations  for  the  QAP,  and  is  closely  related 
to  the  fourth  linearization  presented  in  this  section  due  to  Frieze  and  Yadegar  [75]. 

2.6.1  Lawler’s  Linearization 

Lawler  [123]  replaces  the  quadratic  terms  XijX^i  in  the  objective  function  of 
(2.10),  with  variables 


Vijkl  • b ^ f)  * * * 5 


which  results  in  a 0-1  linear  program  of  -I-  binary  variables  and  + 2n^  + 1 
constraints.  More  specifically,  it  is  proved  [123,  27]  that  the  QAP  is  equivalent  to 
the  following  0-1  linear  program 
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n n 

min  EE  ^ijklVijkl 

ij=l  k,l=l 

s.t.  {xij^  G 

n n 

XI X ^ 

i,j=l  k,l=l 

Xij  ~h  Xj^i  ^Vijkl  ^ 0?  — 1,2,...  ^71^ 

Vijki  e {0,1},  = 1,2,...  ,n. 


The  transformation  of  the  QAP  into  an  equivalent  0-1  linear  program,  will  enable 
its  solution  by  existing  methods  for  linear  integer  programs  and  will  provide  a 
relaxed  problem  such  that  lower  bounds  may  be  computed.  However  the  increased 
number  of  new  variables  and  constraints  usually  poses  an  obstacle  for  efficiently 
solving  the  resulting  linear  integer  programs,  and  a linearization  that  employs  the 
least  number  of  new  variables  and  constraints  is  desirable.  An  even  more  desirable 
property  however  is  the  “tightness”  of  the  relaxation  of  the  resulting  linear  integer 
program. 


2.6.2  Kaufman  and  Broeckx  Linearization 

Rearranging  terms  in  the  objective  function  (2.10)  we  obtain 

n n 

X X ^ijkl^kl' 

ij=l  k,l=l 


Kaufman  and  Broeckx  [112]  defined  new  real  variables 


n 


yjij  . Xij  ^ ^ ^ijkl^kh  j 1,  . . . ) 
/c,Z=l 


resulting  in  an  equivalent  linear  objective  function 
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Introducing  constants  aij  :=  ^ijki  for  hj  = !>  • • • the  QAP  becomes 

equivalent  to  the  following  mixed  0-1  linear  program, 


min  2^  Wij 

S.t,  {Xij^  G 

n 

(^ij^ij  ^ijkl^kl  '^ij  ^ — 1,  . . . , TT,  (2.28) 

k,l=l 

Wij  >0,  z,  j = 1, 2, . . . , n. 


The  above  formulation  employs  new  real  variables,  binary  variables  and 
-I-  2n  constraints.  The  elements  Cijki  are  all  assumed  to  be  nonnegative,  which  is 
a valid  assumption  since  the  addition  of  a constant  to  each  element  will  not  affect 
the  optimal  solution.  The  proof  of  equivalence  of  the  QAP  to  the  above  linear 
integer  program  can  be  found  in  Kaufman  et  al.  [112,  27].  The  above  linearization, 
as  well  as  others  that  appeared  in  the  literature  (see  e.g.  [28,  31]),  are  obtained 
under  the  general  linearization  strategy  as  proposed  by  Glover  [82]. 


2.6.3  Frieze  and  Yadegar  Linearization 

In  Frieze  and  Yadegar  [75]  the  products  of  the  binary  variables  are  replaced 
by  continuous  variables  (i.e.  yijki  :=  XijXki),  and  the  QAP{C)  is  proved  to  be 
equivalent  to  the  following  mixed  0-1  linear  program 


n 71 

min  EE  ^ijklVijkl 

ij=l  k,l=l 

S.t.  ^ -^717 


(2.29) 

(2.30) 
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n 


^ ^ Uijkl  — ^kh  ^ — 1,  . . . , 

i=l 

(2.31) 

n 

^ ^ Uijkl  ^kh  — 1,2,...  ,77/, 

(2.32) 

n 

^ ^ Uijkl  — ^ij^  ^ “ 1,  . . . ,77/, 

k=l 

T1 

(2.33) 

/ V 

^ ^ Uijkl  ^ij 5 ^ — 1,  2,  . . . , 77/, 

1=1 

Uijij  — ^7  J — 1,2,... 

(2.34) 

(2.35) 

0 ^ Uijkl  ^ I7  1,2,...  j Tlj 

(2.36) 

with  new  real  variables,  binary  variables,  and  + 4n^  + + 2n  constraints. 

Note  that  the  constraint  yijij  = Xij  is  redundant  since  it  follows  from  the  definition 
of  the  yijki  variables.  Frieze  and  Yadegar  considered  a Lagrangean  relaxation  of 
the  above  0-1  linear  program,  and  established  a relationship  between  the  lower 
bounds  derived  by  the  solution  of  the  relaxation,  and  the  lower  bounds  derived 
from  decomposition  techniques  applied  to  the  Gilmore-Lawler  bound  for  the  QAP 
(see  §2.9.1). 


2.6.4  Adams  and  Johnson  Linearization 

Adams  and  Johnson  presented  [3]  a new  0-1  linear  integer  formulation  for  the 
QAP,  which  resembles  the  one  of  Frieze  and  Yadegar  described  previously  (see 
§2.6.3).  It  is  based  on  the  general  linearization  technique  for  general  0-1  polynomial 
programs  introduced  by  Adams  and  Sherali  [4,  5].  The  QAP{C)  is  proved  to  be 
equivalent  to  the  following  mixed  0-1  linear  program 


min 

s.t. 


n n 

EE 

i,j=l  k,l=l 

5 


G Xyi 
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n 

^ ^ Vijkl  — ji^il  ~ ^1  • • • ) 

i=l 

n 

^ ^ Vijkl  — ^kl ) i,  /j,  / = 1,  2,  . . . , ?T.,  (2.37) 

j=l 

Vijkl  — Vklij^  J)  ^ ~ 1)  • • • ) 
yijA:/  ^0)  ^ ~ 1 ) 2,  . . . , ?T., 

where  each  yijki  represents  the  product  XijXki-  The  above  formulation  contains  n? 
binary  variables  Xij,  continuous  variables  Vijkh  + 2n^  + 2n  constraints 

excluding  the  nonnegativity  constraints  on  the  continuous  variables.  Although 
as  noted  in  Adams  et  al.  [4]  a significant  smaller  formulation  in  terms  of  both 
the  variables  and  constraints  could  be  obtained,  the  structure  of  the  relaxation 
of  the  above  formulation  is  favorable  for  solving  it.  As  noted  by  the  authors  in 
Adams  et  al.  [3],  the  constraint  set  of  the  above  relaxation  describes  a solution 
matrix  Y which  is  the  kronecker  product  of  two  permutation  matrices  (i.e.  Y = 
X <S>  X where  X € X„),  showing  clearly  the  equivalence  of  the  above  formulation 
with  the  QAP  as  formulated  in  (2.15).  The  theoretical  strength  of  the  above 
linearization  of  the  QAP  lies  on  the  fact  that,  as  the  authors  show  in  Adams 
et  al.  [3],  and  Johnson  [102],  the  constraints  of  the  relaxations  derived  from  all 
previous  linearizations,  can  be  expressed  as  a linear  combination  of  the  constraints 
of  the  continuous  relaxation  of  the  above  linearization.  Moreover,  many  of  the 
previously  published  lower-bounding  techniques,  can  be  explained  based  on  the 
dual-space  of  this  relaxation. 

2.7  Related  Problems 

Generalizations  of  the  QAP  appeared  almost  as  soon  as  the  problem  itself. 
Specifically,  Lawler  [123]  addressed  the  issue  of  extending  to  cubic,  quartic,  and 
M-adic  assignments  problems  in  general,  in  the  same  fashion  as  the  LAP  was 
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extended  to  QAP  in  formulation  (2.10).  For  the  cubic  assignment  problem  for 
example,  we  have  n®  cost  coefficients  where  i,j,k,l,m,p  = 1,...  ,n,  and 

the  problem  is  then  defined  to  be 


min 


n 


n 


n 


EE  E ^ij  klmp^ij  ^kl  ^mp 


ij=l  k,l=l  m,p=l 


S.t  {Xij^  G ^^71  • 

As  it  is  noted  [123],  we  can  construct  an  x matrix  S containing  the  cost 
coefficients,  such  that  the  cubic  assignment  problem  is  equivalent  to  the  LAP 


min  (5,  Y) 
s.t.  Y = X^X®X, 

A ex„. 


In  an  analogous  way  the  LAP  can  be  extended  to  any  A-adic  assignment  problem, 
by  considering  the  solution  matrix  Y to  be  the  kronecker  power  of  a permuta- 
tion matrix  in  X„.  In  this  section  several  generalizations  and  related  problems  of 
the  QAP  are  presented,  for  which  real  applications  have  been  found  that  initiated 
an  interest  in  analyzing  them  and  proposing  solution  techniques. 

2.7.1  The  BiQuadratic  Assignment  Problem 

A generalization  of  the  QAP  is  the  biquadratic  assignment  problem  (BiQAP), 
which  is  essentially  a quartic  assignment  problem  with  cost  coefficients  formed  by 
the  products  of  two  four-dimensional  arrays.  More  specifically,  consider  two  x jY 
arrays  F = {fijki)  and  D = {dmpst)-  The  BiQAP  can  then  be  stated  as: 

nnnnnnnn 

ijkl  dmpst^im^jp^ks^lt 

2=1  j=l  k=l  1=1  m=l  p=l  5=1  t=l 


mm 
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n 


^ij  ^ ^ ) 


j 1,2,...,  71, 

7 = 1,2,...,?!, 

7,  J 1,2,...  , 71, 


The  major  application  of  the  BiQAP  arises  in  Very  Large  Scale  Integrated 
(VLSI)  circuit  design.  The  majority  of  VLSI  circuits  are  sequential  circuits,  and 
their  design  process  is  composed  of  first  translating  the  circuit  specifications  into  a 
state  transition  table,  modeling  the  system  using  finite  state  machines,  and  finally 
trying  to  find  an  encoding  of  the  states  such  that  the  actual  implementation  is  of 
minimum  size.  A detailed  description  of  the  mathematical  modeling  of  the  VLSI 
problem  to  a BiQAP  can  be  found  in  Burkard  et  al.  [41].  Equivalently,  the  BiQAP 
can  be  stated  as: 


min 

pen^v 


n n n n 


/b^'^p(j)p0')p(fc)p(0. 

i=l  j=l  fc=l  1=1 


where  II denotes  the  set  of  all  permutations  of  N = {1,2,...  ,n)  as  usual.  All 
of  the  different  formulations  for  the  QAP  can  be  extended  for  the  BiQAP,  and 
most  of  the  linearizations  that  have  appeared  for  the  QAP  can  also  be  extended 
for  the  BiQAP.  For  example  the  linearization  suggested  by  Adams  and  Johnson 
in  §2.6.4,  can  be  extended  for  the  BiQAP,  by  introducing  continuous  variables 
Vijkimpst  ■=  ^ij^ki^mp^su  ^nd  a Set  of  constraints  on  the  new  variables,  in  addition 
to  the  usual  assignment  constraints,  that  will  ensure  that  the  x matrix  Y 
formed  by  the  Uijkimpst^  is  a kronecker  fourth-power  of  a permutation  matrix  of 
size  77. 

Burkard,  Qela  and  Klinz  [41]  compute  lower  bounds  for  BiQAP  derived  from 
lower  bounds  of  the  QAP.  The  computational  results  showed  that  these  bounds 
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are  weak  and  deteriorate  as  the  dimension  of  the  problem  increases.  This  ob- 
servation suggests  that  branch  and  bound  methods  will  only  be  effective  on  very 
small  instances.  For  larger  instances,  efficient  heuristics,  that  find  good-quality 
approximate  solutions,  are  needed. 

Several  heuristics  have  been  developed  for  the  BiQAP  by  Burkard  and  Qela  [40], 
and  in  particular  deterministic  improvement  methods  and  variants  of  simulated 
annealing  and  tabu  search.  Computational  experiments  on  test  problems  of  size 
up  to  n = 32,  with  known  optimal  solutions  (a  test  problem  generator  is  presented 
in  Burkard  et  al.  [41]),  suggest  that  one  version  of  simulated  annealing  is  best 
among  those  tested.  The  GRASP  heuristic  has  also  been  applied  to  the  BiQAP 
in  [135],  and  produced  the  optimal  solution  for  all  the  test  problems  generated  in 
Burkard  et  al.  [41]. 

2.7.2  Multidimensional  Assignment  Problems 

A close  relative  to  the  class  of  M-adic  assignment  problems  is  that  of  the  mul- 
tidimensional (referred  sometimes  as  multi-index)  assignment  problems  (MAPs), 
that  also  arise  as  natural  extensions  from  the  LAP,  and  the  most  famous  represen- 
tative of  this  class  is  the  3-dimensional  assignment  problem  (3- DAP). 

Consider  three  disjoint  sets  I,  J,  K,  where  without  loss  of  generality  we  can 
assume  that  ]7]  = ]J]  = \K\  = n,  and  cost  coefficients  Cijk-  The  3-DAP  can 
then  be  stated  as 


min 

y^ 

iei  jeJ  keK 

s.t. 

y~^  y~^  ^ijk — 

jeJ  keK 

Vz  G I, 

y^  y~^  ^ijk — 15 

iei  keK 

Vi  G J, 
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^ ^ ^ ^ ^ijk  — 1)  G K, 

iei  jeJ 

Xijk  e {0, 1}  V i,j,k. 


We  can  think  of  Cijk  as  the  cost  of  assigning  job  j to  be  performed  by  worker 
i in  machine  k.  It  follows  that  Xijk  = 1 if  job  j is  assigned  to  worker  i in  machine 
k,  and  Xijk  = 0 otherwise.  A feasible  solution  to  the  above  problem  will  be  a 
three  dimensional  permutation  array,  that  is,  a 0-1  array  that  satisfies  the  above 
assignment  constraints.  The  assumption  |/|  = |J|  = \K\  = n does  not  pose  a 
restriction  theoretically,  since  we  can  add  extra  cost  coefficients  with  large  enough 
values  such  that  they  will  never  be  included  in  the  optimal  solution.  If  we  order 
the  elements  of  the  sets  /,  J,  K and  define  a corresponding  index  set  = 1, . . . ,n 
the  three  assignment  constraints  in  the  above  formulation  of  the  3-DAP  can  be 
interpreted  as:  for  any  pair  Cijk,  cimp  in  the  objective  function  we  should  have  i ^ I, 
j ^ m and  k ^ p.  We  can  equivalently  formulate  the  3-DAP,  by  satisfying  the 
above  requirement  using  permutation  functions  as  follows 


n 


min 

P,Q€Un 


(2.38) 


Pierskalla  [160,  159]  was  the  first  to  make  an  in-depth  description  of  the  3-DAP, 
showing  that  most  of  the  results  obtained  for  the  3-DAP  can  be  extended  to 
the  more  general  MAP.  Various  applications  of  the  3-DAP  are  presented  in  Pier- 
skalla [160,  159].  In  Frieze  [74]  a slightly  generalized  form  of  the  3-DAP  is  consid- 
ered. An  equivalent  bilinear  formulation  of  the  3-DAP  is  given  and  a simple  way 
of  finding  good  but  not  necessarily  optimal  solutions  is  described.  This  is  probably 
the  first  proper  attempt  of  attacking  large  problems  where  integer  programming 
and  branch  and  bound  methods  cannot  be  used.  The  3-DAP  is  shown  to  be  an 
NP-haxd  problem  by  Garey  and  Johnson  [78]  and  Karp  [111]. 
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Many  algorithms  derived  for  the  3-DAP  are  of  branch  and  cut  type  using  the 
facial  structure  of  the  problem,  which  has  been  the  subject  of  study  in  several 
articles.  Balas  and  Saltzman  [15]  examine  the  facial  structure  of  the  three-index 
assignment  polytope  (i.e.  the  convex  hull  of  feasible  solutions  to  the  problem) 
and  identify  two  facets  belonging  to  the  polytope.  0{n‘^)  separation  algorithms 
are  given  for  these.  Balas  and  Qi  [14]  continue  the  above  work  and  present  0{n^) 
separation  algorithms  for  earlier  identified  classes  of  facets  as  well  as  for  a new 
class  (note  that  since  there  are  variables  these  are  linear-time  algorithms).  Qi, 
Balas  and  Gwan  [164]  base  their  work  upon  linear-time  separation  algorithms  for 
diflPerent  facets  and  construct  a polyhedral  procedure  for  solving  the  3-DAP.  The 
general  formulation  of  the  MAP  is 


mm 


s.t. 


Ml  Mj^ 

il  = l 

M‘2  Mjsf 

^ ^ ^ ^ 

22  = 1 2iv  = l 

Ml  Mk-i  Mfc+i  Mn 


E-E  E-E 


X 


ll"‘lN 


= 1, 


n = l 2A;-1  = 1 + *AT  = 1 

for  zjt  = 1, . . . , Mfc  and  k = 2, . . . , A^  — 1 

Ml  Mjv-i 

^ ^ ^ ^ '^N  Ij  . . . j j 

i\  = l Z7v-i=l 

G {0, 1}  for  all  ^ • z;v, 


with  cost  coefficients  As  in  the  3-DAP  a feasible  solution  to  the  above 

problem  will  be  an  A^-dimensional  permutation  array. 

Multidimensional  assignment  problems  in  their  general  form  have  found  many 
applications  as  a means  of  solving  data  association  problems.  More  specifically, 
the  central  problem  in  any  multitarget  tracking  and  multisensor  surveillance  is 
the  data  association  problem  of  partitioning  the  observations  into  tracks  and  false 
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alarms.  General  classes  of  these  problems  can  be  formulated  as  multidimensional 
assignment  problems.  For  a detailed  description  on  the  application  of  MAPs  for 
multiple  target  tracking  applications,  as  well  as  for  a Lagrangian  relaxation  method 
for  solving  these  problems  see  [162]. 

Various  applications  are  also  contributed  to  special  cases  of  the  MAP.  Specifi- 
cally, the  five-dimensional  assignment  problem  has  been  successfully  used  for  track- 
ing elementary  particles.  By  solving  a five-dimensional  assignment  problem,  physi- 
cists reconstruct  tracks  generated  by  charged  elementary  particles  produced  by  the 
large  electron-positron  colider  (LEP)  at  CERN  institute  [163]. 

2.7.3  The  Bottleneck  QAP 

In  the  bottleneck  quadratic  assignment  problem  (BQAP)  of  size  n we  are  given 
an  n X n flow  matrix  F and  an  n x n distance  matrix  D,  and  wish  to  find  a 
permutation  p G 11//  which  minimizes  the  objective  function 


max 


{fijdp{i)pU) 


I <i,j  <n}. 


A more  general  BQAP  analogous  to  the  QAP  in  (2.2)  is  obtained  if  the  coefficients 
of  the  problem  are  of  the  form  Cijku  1 < hF  k,l  < n: 

min  mS/X  ^ijp(i)p(j)  • 

Besides  the  application  in  backboard  wiring  mentioned  above,  the  BQAP  has 
many  other  applications.  Basically,  all  QAP  applications  give  raise  to  applications 
of  the  BQAP  because  it  often  makes  sense  to  minimize  the  largest  cost  instead  of 
the  overall  cost  incurred  by  some  decision.  A well  studied  problem  in  graph  theory 
which  can  be  modeled  as  a BQAP  is  the  bandwidth  problem.  In  the  bandwidth 
problem  we  are  given  an  undirected  graph  G = (V,E)  with  vertex  set  V and 
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edge  set  E,  and  seek  a labeling  of  the  vertices  of  G by  the  numbers  1,2, .. . ,n, 
where  |y|  = n,  such  that  the  minimum  absolute  value  of  differences  of  labels  of 
vertices  which  are  connected  by  an  edge  is  minimized.  In  other  words,  we  seek  a 
labeling  of  vertices  such  that  the  maximum  distance  of  1-entries  of  the  resulting 
adjacency  matrix  from  the  diagonal  is  minimized,  that  is,  the  bandwidth  of  the 
adjacency  matrix  is  minimized.  It  is  easy  to  see  that  this  problem  can  be  modeled 
as  a special  BQAP  with  flow  matrix  equal  to  the  adjacency  matrix  of  G for  some 
arbitrary  labeling  of  vertices,  and  distance  matrix  D = {\i  — j\). 

The  BQAP  is  NP-hard  since  it  contains  the  bottleneck  TSP  as  a special  case. 
(This  is  analogous  to  the  fact  that  the  QAP  contains  the  TSP  as  a special  case,  as 
it  is  shown  in  §2.7.5).  Some  enumeration  algorithms  to  solve  BQAP  to  optimality 
have  been  proposed  by  Burkard  [26].  These  algorithms  employ  a Gilmore-Lawler- 
like  bound  for  the  BQAP  which  involves  in  turn  the  solution  of  bottleneck  linear 
assignment  problems.  The  algorithm  for  the  general  BQAP  involves  also  a thresh- 
old procedure  useful  to  reduce  to  0 as  many  coefficients  as  possible. 

Burkard  and  Fincke  [32]  investigated  the  asymptotic  behavior  of  the  BQAP 
and  proved  results  analogous  to  those  obtained  for  the  QAP:  If  the  coefficients  are 
independent  random  variables  taken  from  a uniform  distribution  on  [0, 1],  then  the 
relative  difference  between  the  worst  and  the  best  value  of  the  objective  function 
approaches  0 with  probability  tending  to  0 as  the  size  of  the  problem  approaches 
infinity. 

The  BQAP  and  the  QAP  are  special  cases  of  a more  general  quadratic  assign- 
ment problem  which  can  be  called  the  algebraic  QAP  (in  analogy  to  the  algebraic 
linear  assignment  problem  (LAP)  introduced  by  Burkard,  Hahn,  and  Zimmer- 
mann  [35]).  If  {H,  *,  -<)  is  a totally  ordered  commutative  semigroup  with  compo- 
sition * and  order  relation  the  algebraic  QAP  with  cost  coefficients  Cijki  € is 
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formulated  as 


min  Ciip(i)p(i)  * Ci2p(i)p(2)  * ...  * Ci„p(i)p(„)  * ...  * ^nnp{n)p{n)  • 

P^^n 


The  study  of  the  bottleneck  QAP  and  more  generally  the  algebraic  QAP  was 
the  starting  point  for  the  investigation  of  a number  of  algebraic  combinatorial 
optimization  problem  with  coefficients  taken  from  linearly  ordered  semimodules, 
that  is,  linear  assignment  and  transportation  problems,  flow  problems,  and  other. 
The  reader  is  referred  to  Burkard  and  Zimmermann  [39]  for  a detailed  discussion 
on  this  topic. 

2.7.4  The  Quadratic  Semi-Assignment  Problem 

In  the  quadratic  semi-assignment  problem  (QSAP)  we  are  given  again  two 
coefficient  matrices,  a flow  matrix  F = (fij)  and  a distance  matrix  D = {dij),  but 
in  this  case  we  have  n “objects”  and  m “locations”,  n > m.  We  want  to  assign 
all  objects  to  locations  and  at  least  one  object  to  each  location  so  as  to  minimize 
the  overall  distance  covered  by  the  flow  of  materials  (or  people)  moving  between 
different  objects.  Thus  the  objective  function  is  the  same  as  that  of  the  QAP, 
and  the  only  different  concerns  the  feasible  solutions  which  are  not  one-to-one 
mappings  (or  bijections)  between  the  set  of  objects  and  locations  but  arbitrary 
functions  mapping  the  set  of  objects  to  the  set  of  locations.  Thus  SQAP  can  be 
formulated  as  follows: 


min 


n n n n n 

^ ^ ^ ^ ^ ^ fijdkl^ik^jl  + ^ ^ bijXij 

i=l  j=l  k=l  l=l  i,i=l 


n 


j=l 


(2.39) 


s.t. 


n. 


(2.40) 

(2.41) 
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SQAP  unifies  some  interesting  combinatorial  optimization  problems  like  clustering, 
m-coloring.  In  a clustering  problem  we  are  given  n objects  and  a dissimilarity 
matrix  F = {fij).  The  goal  is  to  find  a partition  of  these  objects  into  m classes 
so  as  to  minimize  the  sum  of  dissimilarities  of  objects  belonging  to  the  same  class. 
Obviously  this  problem  is  a QSAP  with  coefficient  matrices  F and  D,  where  D is 
an  m X m identity  matrix.  In  the  m-coloring  problem  we  are  given  a graph  with 
n vertices  and  want  to  check  whether  its  vertices  can  be  colored  by  m different 
colors  such  that  each  two  vertices  which  are  joined  by  an  edge  receive  different 
colors.  This  problem  can  be  modeled  as  a SQAP  with  F equal  to  the  adjacency 
matrix  of  the  given  graph  and  D the  mxm  identity  matrix.  The  m-coloring  has  an 
answer  “yes”  if  and  only  if  the  above  SQAP  has  optimal  value  equal  to  0.  Practical 
applications  of  the  SQAP  include  distributed  computing  [180]  and  scheduling  [48]. 

SQAP  was  originally  introduced  by  Greenberg  [89].  As  pointed  out  by  Malu- 
celli  [132]  this  problem  is  NP-hard.  Milis  and  Magirou  [138]  propose  a Lagrangean 
relaxation  algorithm  for  this  problem,  and  show  that  similarly  as  for  for  the  QAP, 
it  is  very  hard  to  provide  optimal  solutions  even  for  SQAPs  of  small  size.  Lower 
bounds  for  the  SQAP  have  been  provided  by  Malucelli  and  Pretolani  [133],  and 
polynomially  solvable  special  cases  have  been  discussed  by  Malucelli  [132]. 

2.7.5  Other  Problems  Which  Can  Be  Formulated  as  QAPs 

There  are  a number  of  other  well  known  combinatorial  optimization  problems 
which  can  be  formulated  as  QAPs  with  specific  coefficient  matrices.  Of  course, 
since  QAP  is  not  a well  tractable  problem,  it  does  not  make  sense  to  use  algo- 
rithms developed  for  the  QAP  to  solve  these  other  problems.  All  known  solution 
methods  for  the  QAP  are  far  inferior  compared  to  any  of  the  specialized  algorithms 
developed  for  solving  these  problems.  However,  the  relationship  between  the  QAP 
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and  these  problems  might  be  of  benefit  for  a better  understanding  of  the  QAP  and 
its  inherent  complexity. 

Graph  partitioning  and  maximum  clique 

Two  well  studied  NP-hard  combinatorial  optimization  problems  which  are  spe- 
cial cases  of  the  QAP,  are  the  graph  partitioning  problem  (GPP)  and  the  maximum 
clique  problem  (MCP).  In  GP  we  are  given  an  (edge)  weighted  graph  G — {V,E) 
with  n vertices  and  a number  k which  divides  n.  We  want  to  partition  the  set  V 
into  k sets  of  equal  cardinality  such  that  the  total  weight  of  the  edges  cut  by  the 
partition  is  minimized.  This  problem  can  be  formulated  as  a QAP  with  distance 
matrix  D equal  to  the  weighted  adjacency  matrix  of  G,  and  flow  matrix  F obtained 
by  multiplying  with  —1  the  adjacency  matrix  of  the  union  of  k disjoint  complete 
subgraphs  with  n/k  vertices  each.  For  more  informations  on  graph  partitioning 
problems  the  reader  is  referred  to  Lengauer  [125]. 

In  the  maximum  clique  problem  we  are  again  given  a graph  G = (V,  E)  with 
n vertices  and  wish  to  find  the  maximum  k < n such  that  there  exists  a subset 
Vi  CV  which  induces  a clique  in  G,  that  is,  all  vertices  of  Vj  are  connected  by 
edges  of  G.  In  this  case  consider  a QAP  with  distance  matrix  D equal  to  the 
adjacency  matrix  of  G and  flow  matrix  F given  as  adjacency  matrix  of  a graph 
consisting  of  a clique  of  size  k and  n — k isolated  vertices,  multiplied  by  —1.  A 
clique  of  size  k in  G exists  only  if  the  optimal  value  of  the  corresponding  QAP  is 
For  a review  on  the  maximum  clique  problem  the  reader  is  referred  to  [156]. 

The  traveling  salesman  problem 

The  traveling  salesman  problem  (TSP)  is  another  well  known  combinatorial 
optimization  problem  which  is  77P-hard,  and  much  research  has  been  devoted  to 
finding  efficient  algorithms  that  will  provide  near-optimal  solutions.  In  the  TSP 
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we  are  given  a set  of  cities  and  the  distances  between  them,  and  our  task  is  to  find 
the  optimal  tour  that  will  visit  each  city  once  and  will  minimize  the  total  distance 
traveled.  More  specifically,  let  the  set  of  integers  N = {1,2, ...  ,n}  represent 
the  n cities  and  the  symmetric  n x n matrix  D — {dij)  represent  the  distances 
between  the  cities,  where  dij  is  the  distance  from  city  i to  city  j,  that  is,  da  = 0 
Vi  = 1, 2, . . . , n.  Then  the  TSP  can  be  formulated  as 


n— 1 

min  E dp(i)p(i-i-i)  T dp(n)p(i) 

i=l 

S.t.  p G IlAf. 


(2.42) 


In  formulating  the  TSP  as  a QAP  the  distance  matrix  D is  the  corresponding 
distance  matrix  of  the  TSP,  and  the  flow  matrix  F is  the  adjacency  matrix  of 
a complete  cycle  of  length  n.  Without  loss  of  generality  the  distance  matrix  D 
is  considered  to  be  symmetric.  A complete  cycle  or  tour  is  then  defined  by  a 
permutation  p. 

Example  2.1  Consider  n = 5 and  p = (3,2,4, 1,5)  then  the  complete  cycle  and 
the  distances  will  be  With  this  in  mind  we  can  construct  the  adjacency  matrix  of  a 


from  city 

to  city 

distance 

3 

2 

ds2 

2 

4 

^24 

4 

1 

dii 

1 

5 

5 

3 

dbz 

complete  cycle  to  represent  the  flow  matrix  F for  n = 5 as  follows: 
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0 10  0 1 
10  10  0 
0 10  10 
0 0 10  1 
10  0 10 


The  optimal  permutation  p*  of  the  QAP,  will  be  the  optimal  tour  of  the  TSP,  and 
the  corresponding  minimum  distance  is  TSP^d,p*)  = 


The  traveling  salesman  problem  (TSP)  is  a notorious  NP-hard  combinatorial  opti- 
mization problem.  Among  the  abounding  literature  on  the  TSP  we  select  the  book 
edited  by  Lawler,  Lenstra,  Rinnooy  Kan  and  Schmoys  [124]  as  a comprehensive 
reference. 


The  linear  arrangement  problem 

In  the  linear  arrangement  problem  we  are  given  a graph  G = (V,  E)  and  wish 
to  place  its  vertices  at  the  points  1,  2,  . . . , n on  the  line  so  as  to  minimize  the 
sum  of  pairwise  distances  between  vertices  of  G which  are  joined  by  some  edge. 
If  we  consider  the  more  general  version  of  weighted  graphs  than  we  obtain  the 
backboard  wiring  problem  (see  §2.1).  This  is  an  NP-hard  problem  as  mentioned 
by  Garey  and  Johnson  [78].  It  can  be  formulated  as  a QAP  with  distance  matrix 
the  (weighted)  adjacency  matrix  of  the  given  graph,  and  flow  matrix  F = (fij) 
given  by  fij  = \i  - j\,  for  all  i,j. 


The  minimum  weight  feedback  arc  set  problem 

In  the  minimum  weight  feedback  arc  set  problem  (FASP)  a weighted  digraph 
G = {V,  E)  with  vertex  set  V and  arc  set  E is  given.  The  goal  is  to  remove  a set  of 
arcs  from  E with  minimum  overall  weight,  such  that  all  directed  cycles,  so-called 
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dicycles,  in  G are  destroyed  and  an  acyclic  directed  subgraph  remains.  Clearly, 
the  minimum  weight  feedback  arc  set  problem  is  equivalent  to  the  problem  of 
finding  an  acyclic  subgraph  of  G with  maximum  weight.  The  unweighted  version 
of  the  FASP,  that  is  a FASP  where  the  edge  weights  of  the  underlying  digraph 
equal  0 or  1,  is  called  the  acyclic  subdigraph  problem  and  is  treated  extensively  by 
Jfinger  [103]. 

An  interesting  application  of  the  FASP  is  the  so-called  triangulation  of  input- 
output  tables  which  arises  along  with  input-output  analysis  in  economics  used  to 
forecast  the  development  of  industries,  see  Leontief  [126].  For  details  and  a concrete 
description  of  the  application  of  triangulation  results  in  economics  the  reader  is 
referred  to  Conrad  [53]  and  Reinelt  [166]. 

Since  the  vertices  of  an  acyclic  subdigraph  can  be  labeled  topologically,  that  is, 
such  that  in  each  arc  the  label  of  its  head  is  larger  than  that  of  its  tail,  the  FASP 
can  be  formulated  as  a QAP.  The  distance  matrix  of  the  QAP  is  the  weighted 
adjacency  matrix  of  G and  the  fiow  matrix  F = (fij)  is  a lower  triangular  matrix, 
that  is,  fij  = —1  if  i < j and  fij  = 0,  otherwise. 

The  FASP  is  well  known  to  be  NP-hard  (see  Karp  [111],  Carey  and  Johnson 
|78J). 

Packing  problems  in  graphs 

Another  well  known  NP-hard  problem  which  can  be  formulated  as  a QAP  is 
the  graph  packing  problem  cf.  Bollobas  [23].  In  a graph  packing  problem  we  are 
given  graphs  G\  = (Vi,Ei),  G2  = {V2,E2)  with  n vertices  each  and  edge  sets  Ei 
and  E2.  A permutation  p of  {1,2,...  ,n}  is  called  a packing  of  G2  into  Gi,  if 
{i,j)  € El  implies  {p{i),p{j))  ^ E2,  for  I < i,  j < n.  In  other  words,  a packing  of 
G2  into  Gi  is  an  embedding  of  the  vertices  of  G2  into  the  vertices  of  Gi  such  that 
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no  pair  of  edges  coincide.  The  graph  packing  problem  consists  of  finding  a packing 
of  G2  into  Gi,  if  one  exists,  or  proving  that  no  packing  exists. 

The  graph  packing  problem  can  be  formulated  as  a QAP  with  distance  matrix 
equal  to  the  adjacency  matrix  of  G2  and  flow  matrix  equal  to  the  adjacency  matrix 
of  Gi . A packing  of  G2  into  G\  exists  if  and  only  if  the  optimal  value  of  this  QAP 
is  equal  to  0.  In  the  positive  case  the  optimal  solution  of  the  QAP  determines  a 
packing. 

2.8  Lower  Bounds 

Lower  bounding  techniques  are  primarily  used  with  implicit  enumeration  algo- 
rithms, such  as  branch  and  bound,  to  perform  a limited  search  of  the  feasible  region 
of  a minimization  problem,  until  an  optimal  solution  is  found.  A more  limited  use 
of  lower  bounding  techniques,  is  also  for  evaluating  the  performance  of  heuristic 
algorithms,  by  providing  a relative  measure  of  proximity  of  the  suboptimal  solution 
to  the  optimum.  In  comparing  lower  bounding  techniques,  the  following  criteria 
should  be  taken  into  consideration: 

• Complexity  of  computing  the  lower  bound 

• Tightness  of  the  lower  bound  (i.e.  closest  to  the  optimum  solution) 

• Efficiency  in  computing  lower  bounds  for  subsets  of  the  primal  feasible  set 

Since  there  is  no  clear  ranking  of  the  performance  of  the  lower  bounds  that  will 
be  discussed  below,  all  of  the  above  criteria  should  be  kept  in  mind  while  reading 
the  following  paragraphs.  Considering  the  asymptotic  behavior  of  the  QAP,  (see 
§2.4)  it  should  be  fair  to  assume  that  the  tightness  of  the  lower  bound  probably 
dominates  all  of  the  above  criteria.  That  is,  if  there  is  a large  number  of  feasible 
solutions  close  to  the  optimum,  then  a lower  bound  which  is  not  tight  enough,  will 
fail  to  eliminate  a large  number  of  subproblems  in  the  branching  process. 
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2.8.1  Gilmore-Lawler  Type  Lower  Bounds 

Based  on  the  formulation  of  the  general  QAP  as  a LAP  of  dimension  stated  in 
formulation  (2.15),  Lawler  [123]  derived  lower  bounds  for  the  QAP,  by  constructing 
a solution  matrix  V in  the  process  of  solving  a series  of  LAPs.  If  the  resulting 
matrix  F is  a permutation  matrix,  then  the  objective  function  value  is  optimal, 
otherwise  it  is  bounded  from  below  by  {C,Y).  Specifically,  consider  an  instance 
of  QAP{C),  where  the  matrix  C is  partitioned  into  minors,  = {cijki)nxn 
for  i,j  = 1,2,...  ,n.  Each  minor  essentially  contains  the  costs  associated 
with  the  assignment  Xij  = 1.  Partition  the  solution  matrix  Y also  into  v?  minors, 
^ for  i,j  = 1,2,...  , n,  whose  actual  values  are  to  be  determined 

in  the  process.  Solve  the  n?  LAPs  associated  with  each  minor 


n n 


^ ^ ^ ^ ^ijklVijkl 

k=l  1=1 
n 

(2.43) 

S.t.  ^ ^ Uijkl  ^ — 1,  2,  . . . , 71, 

k=l 

n 

(2.44) 

^ ^ Vijkl  I5  ^ — 1,2,.  ..,71, 
/=1 

Vijij  ~ 1 

(2.45) 

(2.46) 

Vijkl  ^ — 1,2,...  ,71. 

(2.47) 

Observe  that  the  last  constraint  essentially  reduces  the  problem  into  an  LAP  of 
dimension  (n  — 1),  obtained  by  deleting  the  row  and  column  of  the  matrix 
Denote  the  solution  matrix  for  each  of  the  above  LAPs  by  Fbd)_  Using  the 
values  lij  from  above,  solve  the  LAP  to  obtain  the  Gilmore-Lawler  lower  bound 
for  general  QAPs: 


GLB{C)  = min 


n n 

ij^ij 

i=l  j=l 


{Xij)  G X 


n-) 


(2.48) 


s.t. 
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and  denote  its  solution  matrix  by  X*  = {x^j).  If  the  matrix  \ Y^ij  e X„, 

then  r*  = (a;^F(*^))„2x„2  is  a Kronecker  product  of  two  permutation  matrices 
of  dimension  n,  and  then  it  is  also  an  optimal  solution.  Otherwise  the  optimal 
solution  to  the  QAP  is  bounded  below  by  GLB(C)  = {C,Y*).  Considering  that 
each  LAP  can  be  solved  in  0{n^)  time,  the  above  lower  bound  for  the  general  QAP 
of  dimension  n as  formulated  in  (2.15),  can  be  computed  in  O(n^)  time. 

For  the  more  special  Koopmans-Beckman  formulation  of  the  QAP  (cf.  for- 
mulation 2.1)  where  the  quadratic  costs  are  derived  by  the  pairwise  product  of 
two  matrices  F and  D,  the  structure  of  the  problem  can  be  used  to  reduce  the 
computational  effort.  Before  we  proceed,  let  us  make  some  definitions.  For  vec- 
tors a,b  E IR",  define  the  following  extremal  variations  of  the  usual  inner  product 
between  vectors,  by  imposing  an  ordering  in  the  elements  of  the  vectors: 

n 

{a,b)~  :='^aibi,  where  Oi  > Oi+i,  < 5i+i,  Vz  (2.49) 

i-l 

n 

{a,b)'^  :=\^aibi,  where  Oj  > Oj+i,  > 5i+i,  Vi.  (2.50) 

i=\ 

The  following  well  known  result  is  due  to  Hardy  et  al.  [95]: 

Proposition  2.1  (Hardy,  Littlewood  and  Polya  [95],  1952) 

Fora,  b G IR”  and  N = {1,2, .. . ,n}  the  following  inequalities  hold  for  anyp  G H^v' 


n 

{a,b)~  < '^aibp^i)  < (a,6)+ 
2 = 1 


Consider  an  instance  QAP{F,  D,  B),  and  recall  that  this  can  be  transformed  into 
an  instance  of  QAP{C)  by  assigning  the  values 


for  i^k,j^l, 
for  i = k,  j = 1. 
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Each  minor  in  the  partitioned  matrix  C,  is  now  = f{i,.)dTj  ),  where 
and  ) is  the  and  row  of  matrix  F and  D respectively.  Therefore,  using 
the  result  of  proposition  2.1,  instead  of  solving  n?  LAPs  we  can  easily  compute  the 
values  lij  as 


hj  — fiidjj  + bij , (2.51) 

ys  /-V  ^ 

where  the  vectors  f(i,.),d(^j,.)  G IR"~^  are  obtained  by  removing  the  and 
element  of  the  vectors  /(j_.)  and  d(j^_)  respectively.  Finally  by  solving  the  resulting 
LAP  as  in  (2.48),  we  obtain  the  Gilmore-Lawler  lower  bound  for  the  Koopmans- 
Beckman  QAPs,  denoted  by  GLB{F,  D),  in  0{n^)  time.  Its  name  is  due  to  the 
fact  that  Lawler  [123]  and  Gilmore  [81]  independently  derived  this  lower  bound, 
while  the  first  author  considered  the  case  for  general  QAPs  also. 

Example  2.2  Consider  an  instance  of  QAP{F,  D,  B)  with  no  linear  term  (i.e. 
B = Q),  of  dimension  n = 4 and  input  flow  and  distance  matrices  respectively 
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We  have  to  compute  a matrix  L = (lij)  G IR"^",  where  each  lij  is  the  minimal 
inner  product  between  row  i of  matrix  F and  row  j of  matrix  D with  the  i*'^  and 
elements  removed  respectively,  as  defined  in  (2.51).  Moreover  for  each  Uj  there 
is  an  associated  permutation  matrix  yb4)  q j the  {i,jY^  entry,  and  ones 
in  the  entries  defined  by  the  minimal  inner  product  ordering.  So  for  ^23  = 0 x 10  + 
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2x9  + 3x5  = 33,  while 


y(2,3)  _ 


10  0 0 


0 

0 

0 


0 

0 

1 

0 


1 0 
0 0 
0 1 


Completing  the  entries  of  the  matrix  L = {lij)  we  have 


L = 


57  85  61  80 
31  47  33  44 


33  48  34  45 


62  87  66  83 


To  compute  the  lower  bound  we  have  to  solve  the  LAP  with  cost  matrix  L,  whose 
solution  is 


n n 


GLB{F,D)  = 


min 


UjXij  = 215,  X*  = 


i=\  j=i 


0 0 10 
0 0 0 1 
0 10  0 
10  0 0 


where  X*  is  the  solution  matrix  of  the  LAP.  We  can  also  see  that 


n ^ ^ 4 


10  12 
12  0 1 
0 12  1 
2 110 
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therefore  Y*  — is  not  a Kronecker  produet  of  two  permutation  matrices, 

and  we  don’t  have  a feasible  solution.  Note  that  the  optimal  value  for  this  example 
is  217. 

The  simplicity  of  the  Gilmore-Lawler  lower  bound  makes  it  one  of  the  most  ef- 
ficient to  compute,  although  it  deteriorates  fast  as  n increases.  The  quality  of 
the  lower  bound  can  be  improved  if  the  contribution  of  the  quadratic  term  in  the 
objective  function  is  made  to  be  smaller  than  that  of  the  linear  term.  Consider 
the  formulation  of  the  general  QAP  as  presented  in  (2.11),  where  the  linear  and 
the  quadratic  terms  are  separated  for  clarity.  By  the  above  discussion  the  lower 
bound  will  be  the  solution  to  the  LAP 

n n 

min  ^ ] 'y  + Cijij)xij 

i=l  j=l 

s.t.  {xij)  6 X„.  (2.52) 

We  want  to  decompose  the  cost  coefficients  in  the  quadratic  term  of  (2.11)  and 
transfer  some  of  their  value  into  the  linear  term  such  that  Cijij  » /y,  which  will 
result  in  a tighter  lower  bound  since  the  LAP  can  be  solved  exactly.  This  procedure 
is  known  as  reduction  was  introduced  by  Conrad  [53],  and  it  has  been  investigated 
by  many  researchers  (see  [64,  75,  173,  25]).  The  general  idea  is  to  decompose  each 
quadratic  cost  coefficient  into  several  terms,  which  in  turn  will  end  up  being  linear 
cost  coefficients  and  will  be  moved  in  the  linear  term  of  the  objective  function. 
Consider  the  following  general  decomposition  for  each  quadratic  cost  coefficient  in 
the  objective  function  in  (2.11) 


D"l"  Cjjfc!  — Qjfci  T ^ijk  T 9iji  ”b  hiki  T tji^i,  i ^ k,  j ^ I, 
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3 

where  e,  g,h,t  G IR"  . 


Substituting  the  above  and  separating  terms,  the  objective 


function  in  (2.11)  becomes 


n 


n 


n 


n 


n 


n 


n 


^ V ^ V ^ V “1“  ^ ^ ^ ^ ^ ^ (fiijk  + 9ijl  “l~  ^ikl  tjkl)^ij^kl  "1“  ^ ^ 


^ijij  ^ij 


1=1  j=i  A:,/=l 
i^k  j^l 


i = l 

i^k 


j=i  k.l- 


i,j=^ 


Consider  now  the  term  associated  with  the  e^-fc 


n n n 

n n n 

n 

n 

( n \ 

— ^ ^ ^ ^ ^ ^ ^ijk^ij^kl 

^ V ^ijk 

i = l j = i k,l  = l 

ij  = l k = l 1 = 1 

i,j=l 

k=i  ' 

1=1  1 

^^k  j^l 

_ k^i 

\ / 

We  can  add  the  term  X)"=i  Cijk  to  the  {i^jY^  element  of  the  LAP  that  composes  the 

k^i 

linear  term  of  the  objective  function,  since  Xij  = 1 Xkj  = 0 =>  = 1,VA:. 

Using  similar  arguments  for  the  vectors  g,  h and  t,  their  costs  become  linear  and 
the  objective  function  with  decomposed  quadratic  costs  become 


n n n n 


i=i  i=i  k—\  /=1 


n n 


^ijXij , 

i=l  j=l 


where 


n 


fc  = l 1 = 1 k = l 1 = 1 

k^i  l^j  k^i  l^j 


Therefore  we  can  apply  the  Gilmore-Lawler  in  the  quadratic  term  of  the  decom- 
posed objective  function,  whereas  we  can  get  an  exact  solution  to  the  LAP  that 
composes  the  linear  term,  and  the  sum  of  these  two  values  will  constitute  a lower 
bound  for  the  QAP.  In  the  case  of  the  Koopmans-Beckman  formulation  of  the 
QAP  were  we  have  two  matrices  F and  D,  the  general  decomposition  scheme  is 
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D"2.  fij  fij  + + /ij,  ^ 

dki  dki  ~h  l^k  Pi  j ^ 7^  ^7 


where  X,fi,u,p  G IR^.  Substituting  to  the  product  fijdki  it  is  easily  seen  that  D-2 
reduces  to  the  general  decomposition  D-1  with  vectors  e,g,h,t  G H"  . Frieze  and 
Yadegar  [75]  showed  that  the  inclusion  of  the  vectors  h and  t in  the  D-1  does  not 
affect  the  value  of  the  lower  bound,  and  therefore  are  redundant  (similarly  the 
vectors  /x  and  p for  the  Koopmans-Beckman  QAP  are  redundant  in  D-2).  The 
same  authors  in  Frieze  et  al.  [75]  derived  lower  bounds  for  the  QAP  based  on  a 
Lagrangean  relaxation.  Specifically  consider  the  Lagrangean  relaxation  of  the  0- 
1 linear  programming  formulation  of  the  QAP  as  described  in  §2.6.3,  where  the 
constraints  (2.31)  and  (2.32)  are  included  in  the  objective  function  (2.29),  using 
as  Lagrangean  multipliers  the  elements  of  the  vectors  e and  g.  The  Lagrangean 
function  is  thus  defined  as 


-C(e,(/)  = 


min  E ^ijklVijkl  ^ ^ ^jkl  I ^kl  ^ ^ Vijkl  I ^ ^ 9ikl  | ^kl  ^ ^ Vijkl 
ijkl  jkl  \ i J ikl  \ j 

~ ^jkl  ~ 9ikl)Uijkl  "b  ( X/  + x^ 

ijkl  ij  \ k I 


Qlij  I ^ij 


s.t.  constraints  (2.30),  (2.33). . . (2.36). 


The  authors  prove  in  Frieze  et  al.  [75]  that  for  any  choice  of  e and  g,  the  solu- 
tion to  the  above  minimization  problem  will  equal  the  value  of  the  Gilmore-Lawler 
lower  bound  as  applied  to  the  QAP,  with  decomposed  quadratic  cost  coefficients, 
as  dictated  by  using  the  vectors  e and  g only  in  D-1.  Therefore,  maxg.p  £(e,  g)  con- 
stitutes an  upper  bound  on  the  lower  bounds  for  the  QAP,  obtained  by  using  the 
Gilmore-Lawler  bound  with  decomposed  quadratic  cost  coefficients.  Using  sub- 
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gradient  algorithms  the  authors,  derive  near  optimal  solutions  for  maxe,g£(e,  5') 
resulting  in  two  lower  bounds,  denoted  by  FYl  and  FY2,  corresponding  to  the 
two  different  solution  approaches  proposed.  As  suggested  by  the  experimental 
results  in  Frieze  et  al.  [75],  these  bounds  seem  to  be  sharper  than  previously  re- 
ported Gilmore-Lawler  based  lower  bounds  using  reduction  techniques.  Almost 
all  of  the  other  approaches  for  obtaining  lower  bounds  for  the  QAP  with  reduc- 
tion techniques,  are  special  cases  of  the  general  decomposition  scheme  D-2  (see 
[64,  173,  25]). 

2.8.2  Variance  Reduction  Lower  Bounds 

The  variance  reduction  lower  bounds  were  introduced  by  Li,  Pardalos,  Ra- 
makrishnan  and  Resende  [128].  Consider  an  instance  of  the  Koopmans-Beckman 
formulation  of  the  QAP,  with  input  matrices  F = {fij),D  = (dij)  G IR"^".  Now 
partition  both  matrices  into  a sum  of  two  matrices,  F = F1  + F2  and  D = D1+D2, 
were  Fi  = (fip),  F2  = (fif)  and  Di  = D2  = (d\f).  Construct  a n x n 

matrix  L = (kj),  by  solving  the  following  n?  LAPs  (cf.  problem  defined  in  (2.43)) 

~ ^ki^p{k)j  + fkidp(k)j  ~ -l^ki^ ^p{k)]^  (2.53) 

p(i)=j  k=l 

It  is  proved  in  Li  et  al.  [128]  that  the  solution  of  the  LAP  with  cost  matrix  L 
as  constructed  above,  constitutes  a lower  bound  for  QAP{F,  D).  The  problem  of 
concern  now,  is  to  find  a way  to  partition  the  matrices  F and  D such  that  the 
resulting  lower  bound  is  maximized.  Observe  that  when  Fi  — F and  D\  — D (i.e. 
no  partitioning),  we  essentially  derive  the  GLB{F,  D). 

Let  M G IR"*^",  and  denote  its  rows  and  columns  respectively  as  and 

m(.j),  i,j  = 1, . . . ,n.  Think  of  M as  a data  set  of  mn  elements  rriij,  and  define 
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an  average  'y(M)  and  a variance  V{M)  as 

^ m n m n 

■=  ■=  EEh(^)  -"“i)'- 

j=l  j=l  i=l  j=l 

Also  define  the  total  variance 

m 

T{M,  A)  :=  A J]V'(mp,.,)  + (l-A)y(M),  A 6 [0,1], 

i=l 

Where  the  term  V (m(i^.))  stands  for  the  variance  of  treated  as  an  1 xn  matrix. 
The  authors  observed  that,  as  the  variances  of  the  matrices  F and  D decrease,  the 
GLB{F,  D)  increases,  while  it  is  optimal  if  the  variances  of  the  rows  of  the  matrices 
are  zero.  The  partition  scheme  considered  is  of  the  form,  Fi  = F + Ap,  F2  = —Ap, 
and  Di  — D + Ap>,  D2  = —Ap.  Considering  only  the  matrix  F,  the  problem  is 
to  find  a matrix  A^-,  such  that  the  variances  of  Fi  and  F2  and  the  sum  of  the 
variances  of  the  rows  for  each  Fi  and  F2  are  minimized.  We  will  only  describe  how 
Af  is  obtained  since  Ax?  is  obtained  in  the  same  way.  The  problem  of  minimizing 
the  variances  can  be  stated  mathematically  as 

min  eT{F  + Ap,X)  + {l-e)T{-Al,X),  (2.54) 

where  Ap  G IR”^”  and  6 G [0, 1]  is  a parameter.  Two  approximate  solutions  were 
proposed  in  Li  et  al.  [128],  corresponding  to  the  two  reduction  schemes 

R-1.  Sij  ^(/nn  fij')  T b .)  — 1,  . . . , U, 

R-2:  5ij  = 0(7(/(.,n))  - 7(/(.,j))  b i = T • • • , «, 

where  Ap  = {Sij),  and  with  being  free  to  take  any  value  (it  was  given  a value 
of  zero  in  the  experiments  conducted  in  Li  et  al.  [128]).  In  obtaining  R-2,  the 
problem  of  minimizing  the  variances  such  that  the  matrix  A;?  is  constrained  to 
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have  constant  columns,  is  considered.  The  matrix  Ad  is  constructed  in  the  same 
way.  Based  on  the  two  reductions  schemes  above,  the  resulting  lower  bounds  from 
the  solution  of  (2.53)  are  denoted  by  LB1{9),  and  LB2{9).  The  above  procedure 
for  computing  Af,Ad  has  O(n^)  computational  complexity. 

After  the  partitioning  of  the  matrices  F and  £>,  the  solution  to  the  LAP 
with  cost  matrix  L = {kj),  were  kj  are  defined  in  (2.53),  will  yield  LB1{9)  or 
LB2{9)  according  to  what  reduction  scheme  used.  If  LB2{9)  is  used,  the  fact  that 
the  matrices  F-i  and  D2  have  constant  columns  can  be  exploited  to  compute  the 
hj  = 1, . . . , u,  efficiently  as 


I- 


( ) 


+ /!? 


n 


n 


fk: 


-{n- 


A:  = l 
^^3 


k=i 

k^i 


where G H”-  are  the  and  row  of  Fi  and  Di  respectively,  with  the 
and  elements  removed  from  each,  and  (•,  •)“  is  defined  in  (2.49).  In  the  case 
that  LB1{9)  is  used,  the  direct  approach  would  be  to  solve  the  n?  LAPs  defined 
in  (2.53),  which  will  require  0{n^)  computational  effort.  A different  approach  is 
to  calculate  lower  bounds  for  the  values  kj,  = 1, . . . , n,  as  follows 


where  each  vector  in  the  above  extremal  inner  products,  is  of  dimension  n — 1,  and 
corresponds  to  the  row  or  column  of  the  indicated  matrix,  upon  removal  of  the 
element.  Similarly  as  before  the  extremal  inner  products  (•,  •)“  and  (•,  •)■*■  are 
defined  in  (2.49)  and  (2.50).  Using  the  above  approach  would  require  0{n^)  time 
to  compute  lower  bounds  for  the  i,j  = 1, . . . ,n,  thus  the  total  computational 
complexity  of  the  variance  reduction  lower  bounds  is  O(rz^).  It  is  worth  noting 
that  there  is  also  a closed  form  solution  to  problem  (2.54)  given  by  Jansen  [100] 
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which  is 


Sij 


- «>  - - O'! 


1-ex 

9X{1  - 9) 
I -X  + 9X 


{I  - 9X){\  - X + 9X) 


7(F) 


7(/(.j))  - Ofij, 


for  i,j  = 1,...  ,n.  However  it  was  reported  in  Li  et  al.  [128]  that  using  the 
above  closed  form  in  the  computation  of  the  lower  bounds,  poses  implementation 
obstacles.  The  experimental  results  conducted  in  Li  et  al.  [128],  also  suggest  that 
the  settings  oi  9 = 0.5  for  LB1{9),  and  0 = 1 for  LB2{9)  as  best  choices.  Finally, 
these  lower  bounds  perform  well  on  QAPs  with  input  matrices  that  have  high 
variances,  but  their  performance  reduces  to  that  of  the  Gilmore-Lawler  bounds 
when  the  variance  of  the  matrices  is  small. 

2.8.3  Eigenvalue  Based  Lower  Bounds 

These  bounds  were  introduced  by  Finke  et  al.  [70,  71],  and  are  applied  to  the 
Koopmans-Beckman  formulation  of  the  QAP  presented  in  (2.1).  This  approach 
utilizes  known  results  on  permutation  matrices  and  eigenvalues,  and  exploits  the 
special  structure  of  QAP{F,  D).  Upon  the  introduction  of  the  method  by  Finke 
et  al.  [70,  71],  many  improvements  and  generalizations  have  appeared  (see  for 
example  [167,  90,  92,  91,  93,  168]).  There  is  a resemblance  with  the  Gilmore- 
Lawler  based  lower  bounds,  in  the  sense  that,  based  upon  a general  lower  bound, 
reduction  techniques  are  applied  to  the  quadratic  terms  of  the  objective  function  in 
order  to  improve  its  quality.  The  reduction  techniques  that  applied  to  eigenvalue 
based  lower  bounds  however,  yield  a significant  improvement,  which  is  not  really 
the  case  with  the  Gilmore-Lawler  bounds  under  certain  reductions. 
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Considering  the  trace  formulation  of  the  QAP  in  (2.13),  with  F and  D being 
real  symmetric  matrices  (cf.  §2.2.4),  therefore  with  all  their  eigenvalues  being  real, 
the  following  result  can  be  stated  for  the  quadratic  term  [71]: 

Theorem  2.9  (Finke,  Burkard,  Rendl  [71],  1987) 

Let  F,D^  be  symmetric  matrices,  and  denote  by  X = (Aj,...  , A„)^  and 

Xi,...  ,Xn  the  eigenvalues  and  eigenvectors  of  F,  and  by  /a  = (/ii,...  cind 

y\,. . . ,yn  the  the  eigenvalues  and  eigenvectors  of  D.  Then  the  following  two  rela- 
tions are  true  for  all  X G X„, 

(i)  trFXDX-^  = E”,,  Xy,y  = A^S(X)^ 

where  S{X)  = {{xi,  Xyj)^)  is  a doubly  stochastic  matrix, 

(ii)  (A,  fi)~  < trFXDX^  < (A,  /r)+. 

Using  part  (ii)  of  the  theorem  2.9  a lower  bound  for  QAP{F,D)  based  on  the 
eigenvalues  of  F and  D is  then 

EVB  = (A,/x)“+  min  trBX^, 

xex„ 

where  the  second  term  is  an  ordinary  LAP  that  can  be  solved  exactly.  Observe 
that  in  theorem  2.9,  the  smaller  the  interval  [(A,  //)“,  (A,  /x)+]  is,  the  closest  (A,  ii)~ 
is  to  trFXDX^.  A possible  way  of  making  the  interval  smaller,  is  to  decompose 
the  matrices  F and  D such  that  some  of  their  value  will  be  transfered  in  the  linear 
term,  and  the  eigenvalues  of  the  resulting  matrices  that  compose  the  quadratic 
term,  are  as  uniform  in  value  as  possible.  Define  the  spread  of  the  matrix  F as 


spread{F)  :=  max  { | Aj  — Xj 


i, j 1,...  ,7i)‘. 
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Based  on  the  above  discussion,  we  want  to  minimize  the  spreads  of  the  matrices 
that  compose  the  quadratic  term.  There  is  no  simple  closed  form  for  expressing 
spread{F)  in  terms  of  fij,  however  we  can  minimize  instead  a formula  for  the  upper 
bound  given  by  Mirsky  [140] 


spread{F)  < m{F)  = 


n n 


1 1/2 


. i=l  j=l 


(2.55) 


The  decomposition  scheme  that  the  authors  use  in  Finke  et  al.  [71],  is  the  following 


fij  — fij  + Cj  + 6j  + rij,  (2.56) 

dki  = df-i  + Qk  + 9i,  +Ski,  (2-57) 

where  = 0,  for  i 7^ 

Consider  the  decomposition  for  matrix  F and  let  F = {fij).  Minimizing  the 
function  f{e,r)  = m{F)  obtained  by  substituting  the  values  of  fij  in  (2.55),  the 
following  values  are  obtained  [71] 


1 


n n 


2(n-  1) 


fij  - trF 


i=l  j=l 


1 


n 


= 


n — 2 


^ ^ fij  fa  ^ — 1,  . . . , 71, 


j=l 


n 


fa  2ej,  z 1, . . . , 7z. 


(2.58) 

(2.59) 

(2.60) 


Analogously  we  obtain  the  values  for  g and  s for  the  decomposition  of  D.  Replacing 
F and  D in  (2.13)  we  obtain 

tr{FXD  + B)X^  = tr{FXD  + B)X^, 
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wher6  bij  — b{j  + fudjj  + 2ei  i:  k = l djj^^  and  matrices  F and  D have  respective 

kj^j 

eigenvalues  A = (Ai, . . . , A„)  and  p,  = {pi, . . . , p^).  The  corresponding  eigenvalue 
lower  bound  is  then 

EVB\  = C\,p)~  + min  trBX^. 

X€Xn 

Example  2.3  Consider  the  problem  of  computing  EVBl  for  the  QAP  instance 
examined  in  example  (2.2).  Since  matrix  F is  not  symmetric,  we  have  to  replace 
it  with  a symmetric  matrix  E where  (see  ^2.2.4) 

0 7 19 
7 0 3 1 
13  0 7 
9 17  0 

Now  we  have  the  compute  the  vectors  e = (ei,...  ,64)^,  r = {ru,...  ,r^4)^  for 
matrix  E,  and  g = {gi, . . . ,54)^,  s = (sn, . . . , 544)^  for  matrix  D,  as  defined  in 
(2.59)  and  (2.60)  respectively.  Computing  the  above  we  have 


£=1(F  + J=’’')  = i 


6=^(115,25,25,115)^,  r = — 2e  and  ^ = ^(16, 37, 19, 34)^,  s = —2g. 

Applying  the  decomposition  scheme  defined  in  (2. 56),  (2. 57)  for  matrices  E and  D 
we  get 


0 

7 

-11 

4 

0 

7 

-5 

-2 

1 

7 

0 

4 

-11 

1 

7 

0 

-2 

-5 

^=6 

-11 

4 

0 

7 

-5 

-2 

0 

7 

4 

-11 

7 

0 

-2 

-5 

7 

0 
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and 


8 

115 

92 

111 

19 

25 

20 

24 

19 

25 

20 

24 

88 

115 

92 

111 

where  the  new  matrix  B is  the  linear  term  introduced  in  the  objective  function  by 
the  decomposition  of  E and  D.  The  spectra  of  E and  D are  respectively  . 


-0.19081  X 10■^ 


-0.46983  X 10~®, 


8 Ml 

6’TJ 


where  if  treated  as  vectors  in  give  (X,p)~  = -13.33334.  The  solution  of  the 
LAP  with  cost  matrix  B is  229,  therefore  EVBl  = 229  — 13.33334  216.  Note 

that  applying  the  eigenvalue  bound  without  decomposition  would  result  in  a lower 
bound  of  -253.  In  fact  EVBl  may  be  negative  for  some  instances  of  QAPs. 

If  we  restrict  ourselves  only  to  pure  quadratic  {fa  = da  = 0,  Vz,  5 = 0)  symmet- 
ric QAPs,  the  matrix  B in  the  above  decomposition  becomes  B = cw^,  where 

c = 2(ei, . . . , e„)^  and  w = (Y2j  dnjY ■ Therefore  min^eXn  trBX'^  = 

(c,  w)~ , and 

EVBl  = {Xfi)- + {c,w)~  < mmtr{FXDvB)X^. 

X.  GXn 

We  can  however  obtain  further  improvement  as  Rendl  [167]  suggested,  where  he 
examined  the  linear  term  {c,w)~,  and  proposed  a method  where  EVBl  is  itera- 
tively improved,  until  some  specified  number  of  iterations  is  reached,  or  we  have 
satisfied  an  optimality  condition.  More  specifically,  let  Sk  :=  {Xi, . . . ,Xk}  C X„, 


67 


and 

L{Xi)  :=  min  {(c,  Xiw)  | X,  e X„  \ 5i_i}  , 

so  for  any  integer  A:  > 1,  L{Xi)  < L{X2)  < •••  < L{Xk).  In  other  words 
the  set  Sk  contains  the  k first  solutions  (permutation  matrices)  of  the  problem 
minxex„(c,  Xite),  where  the  first  solution  X\  3 L{X\)  = {c,w)~.  Let 


QAP{F,  D,  Xi)  = tr{FXiD  + B)Xj , 

and  also  define  the  following 

Z{k)  :=  min  {QAP{F,  D,  Xi)  | i = 1, . . . ,k}  . 

The  following  inequalities  [167]  result 

Z{1)  > • • ■ > Z{k)  > (A,  p)  + L{Xk)  > • • • > (A,  /l)  + L{Xi), 

where  if  Z{i)  = (X,p)~  + L{Xi)  for  some  i,  then  Xi  is  the  optimal  solution  to  the 
problem.  So  essentially,  we  try  to  close  or  reduce  the  gap  between  the  optimal 
solution  of  the  QAP  and  the  lower  bound  EVBl,  by  increasing  the  value  of  the 
linear  term  {c,w)~  in  the  bound  in  k steps,  where  k is  specified  as  a parameter. 
The  generation  of  the  set  Sk  or  ranking  as  it  is  called,  is  a special  case  of  the 
problem  of  ranking  the  k first  solutions  of  an  assignment  problem  with  cost  matrix 
(ciWj)  where  as  Murty  [146]  has  shown  has  time  complexity  0{kn^).  Rendl  [167] 
presents  an  0(nlogn  + (n  + logA:)A:)  for  this  special  case.  There  are  two  issues 
regarding  the  effectiveness  of  the  above  ranking  procedure,  in  improving  the  lower 
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bound,  addressed  by  Rendl  [167],  First,  observe  that  if  the  vectors  c and  w have 
m < n equal  elements,  then  there  are  at  least  m\  permutation  matrices  {Xj}  such 
that  the  values  (c,  Xiw)  are  equal.  This  in  turn,  implies  that  there  will  be  none  or 
small  improvement  in  the  lower  bound  while  generating  Sk  for  quite  some  number 
of  iterations.  As  dictated  by  the  decomposition  in  (2.58),  (2.59),  c and  w will  have 
equal  elements  if  the  row  sums  of  F and  D are  equal.  One  condition  then  for 
applying  the  ranking  procedure,  is  that  most  of  the  row  sums  of  F and  D are  not 
equal.  Secondly,  Rendl  [167]  also  defines  a ratio  called  the  degree  of  linearity  based 
on  the  ranges  of  the  quadratic  and  linear  terms  that  compose  the  lower  bound 

_ (A,//)+  - (X,p.)- 
{c,w)+ - {c,w)-' 

The  influence  of  the  linear  term  on  the  lower  bound  then  is  inversely  proportional 
to  the  value  of  L.  A small  value  of  L suggests  that  the  ranking  procedure  would  be 
beneficial  for  the  improvement  of  EVBl  for  symmetric,  pure  quadratic  QAPs.  For 
large  values  of  L,  we  can  expect  that  the  quadratic  term  dominates  the  linear  term 
in  the  objective  function,  and  Finke  et  al.  [71]  suggest  the  following  improvement 
on  EVBl.  Considering  part  (i)  of  theorem  2.9  as  applied  to  the  reduced  matrices 
F and  D,  denote  the  elements  of  the  matrix  S{X)  by  Sij  = {xi,Xyj)‘^.  We  can 
apply  the  bounds  lij  < Sij  < Uij  where 

Uij  = max{{{xi,yj)~)^,  ((xi,%)+)^}. 


min{((a;i,  yj)  )^,  {{xi, 

\ 


if  {xi,yj)  ,{xi,yj)'^  different  in  sign, 
otherwise. 
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Recalling  the  fact  that  the  Sij  are  the  elements  of  a doubly  stochastic  matrix,  we 
can  then  form  the  capacitated  transportation  problem 


CTP*  = min 


s.t. 


n n 


i=l  j=l 


i=i 

lij  ^ S{j  ^ * 


,n, 

,n, 


The  new  lower  bound  then  would  be 

EVB2  = CTP*  + {c,w)~. 


A more  generalized  approach  to  eigenvalue  based  lower  bounding  techniques, 
was  employed  by  Hadley,  Rendl  and  Wolkowicz  [91],  that  led  to  new  lower  bounds. 
Consider  the  following  sets  of  n x n matrices,  where  I G is  the  identity 

matrix,  and  , 1)^  G IR"  is  the  vector  of  ones. 


0:={X\  X^X  = /},  set  of  orthogonal  matrices, 

S {X  I Xu  = X^u  = u},  set  of  matrices  with  row 

and  column  sums  equal  to  one, 
M :=  {X  I X > 0},  set  of  nonnegative  matrices. 


(2.61) 


It  is  a well  known  result  that  X„  = 0 n £ D Af,  while  the  set  of  doubly  stochastic 
matrices  = £ fl  N . Moreover,  by  Birkhoff’s  theorem  [21]  we  know  that  is 
a convex  polyhedron  with  a vertex  set  X„,  that  is,  = conv{X  ; X G X„}. 
Considering  the  above  characterization  of  X„,  we  can  see  that  any  solution  to  a 
relaxation  of  the  QAP  obtained  from  excluding  one  or  two  of  the  matrix  sets  O,  £ 
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and  jV,  will  yield  a lower  bound.  Naturally  the  relaxation,  and  therefore  the  lower 
bound,  will  be  tighter  if  only  one  of  the  matrix  sets  is  excluded.  In  relation  to 
theorem  2.9,  Rendl  and  Wolkowicz  [168]  showed  that 


min  trFXDX^  = 
xeo 


trFApAlDADA'^ 


maxtrFX  DX^ 
xeo 


tvFAFAlDADAl=  (A,/x)+, 


where  Af,Ad  are  the  matrices  with  columns  the  eigenvectors  of  F and  D re- 
spectively, in  the  order  specified  by  the  minimal  (maximal)  inner  product  of  the 
eigenvalues.  In  other  words,  the  lower  bound  on  the  quadratic  part  of  the  QAP 
as  obtained  from  the  EVB,  is  derived  by  relaxing  the  feasible  set  to  that  of  or- 
thogonal matrices.  In  Rendl  et  al.  [168]  a new  lower  bound  is  derived,  similar  to 
EVB2  but  using  a different  approach  to  decompose  the  matrices  F and  D.  More 
specifically,  denote  the  decomposition  scheme  in  (2.56)  and  (2.57)  by  the  vector 
d :=  r^,  s^)  G IR^",  where  r = (rn, . . . , r„„)^  and  s = (sn, . . . , SnnY ■,  and 

consider  EVB\  as  a function  of  d.  Maximizing  this  function  with  respect  to  d will 
result  in  a lower  bound  with  the  best  possible  decomposition  that  involves  both 
the  linear  and  quadratic  terms.  This  leads  to  a nonlinear,  nonsmooth,  nonconcave 
function  which  is  hard  to  solve,  and  a steepest  ascent  algorithm  is  proposed  for 
maximizing  it  in  Rendl  et  al.  [168].  The  new  bound,  denoted  EVB3,  produces 
some  of  the  best  lower  bounds  for  the  QAP,  with  the  expense  however  of  high 
computational  requirements. 

All  of  the  above  discussed  lower  bounds,  relax  the  set  of  permutation  matrices 
to  O.  A tighter  relaxation  was  proposed  by  Hadley  et  al.  [92],  where  the  set  of 
permutation  matrices  was  relaxed  to  O C\  S,  by  incorporating  S in  the  objective 
function,  by  exploiting  the  fact  that  the  vector  of  ones  u is  both  a left  and  right 
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eigenvector  with  eigenvalue  1,  for  any  X G X„.  More  specifically  define 


P :=  [u/ 


u 


where  V^u  = 0,  V^V  = /„_i, 


therefore  V is  an  orthonormal  basis  for  {u}-*-,  while  Q :=  VV'^  is  the  orthogonal 
projection  on  {«}■'■.  The  following  characterization  of  the  permutation  matrices  is 
given  in  Hadley  et  al.  [92] 

Lemma  2.1  (Hadley,  Rendl,  Wolkowicz  [92],  1992) 

Let  X e and  Y G If 


then 


X = P 


1 0 

0 y 


(2.62) 


X es, 


X VYV^  > -uu^j 


X^O^Y  ^ On-\. 


Conversely  if  X € £ then  there  exists  a Y such  that  (2.62)  holds. 

Note  that  the  above  characterization  of  the  permutation  matrices,  preserves  the  or- 
thogonality and  the  trace  structure  of  the  problem.  Substituting  X = —uu'^/\\u\\‘^+ 
VYV^  as  suggested  by  (2.62),  in  the  trace  formulation  of  the  QAP  in  (2.13),  we 
have  an  equivalent  projected  problem  (PQAP)  of  dimension  n — 1 on  the  vari- 
able matrix  Y.  The  new  lower  bound  IV B is  obtained  by  relaxing  Y to  0„_i, 
therefore  deriving  a lower  bound  for  the  quadratic  part  of  PQAP,  while  the  linear 
part  can  be  solved  exactly  as  an  LAP.  Decompositions  for  improving  the  IV B are 
also  considered  in  Hadley  et  al.  [92],  where  it  is  shown  that  the  quadratic  term 
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in  the  projected  problem  is  unaffected  by  e and  g in  the  decomposition  scheme 
in  (2. 56), (2. 57).  Obtaining  a lower  bound  by  considering  both  the  quadratic  and 
linear  term  is  also  considered  by  Karisch  et  al.  [109]. 

The  symmetry  assumption  on  the  QAP  is  required  by  any  of  the  eigenvalue 
based  lower  bounding  techniques  described  above.  Hadley,  Rendl  and  Wolkow- 
icz  [93]  show  that  any  real  QAP  can  be  transformed  into  an  equivalent  QAP  where 
the  matrices  F and  D are  Hermitian,  which  allows  the  application  of  eigenvalue 
based  bounds. 

2.8.4  Bounds  Based  on  Semidefinite  Relaxations 

Semidefinite  programming  (SDP)  is  a generalization  of  linear  programming 
where  the  variables  are  taken  from  the  Euclidean  space  of  matrices  with  the  trace 
operator  acting  as  an  inner  product.  The  non-negativity  constraints  are  replaced 
by  semidefiniteness  constraints  and  the  linear  constraints  are  formulated  in  terms 
of  linear  operators  on  the  above  mentioned  Euclidean  space  of  matrices.  Successful 
applications  of  semidefinite  programming  in  discrete  optimization  are  presented  in 
Goemans  and  Williamson  [86],  and  Lovasz  and  Schrijver  [131]. 

Recently,  semidefinite  programming  relaxations  for  the  QAP  were  considered 
by  Karisch  [107],  Zhao  [187],  and  Zhao,  Karisch,  Rendl  and  Wolkowicz  [110].  The 
SDP  relaxations  considered  in  these  papers  are  solved  by  interior  point  methods 
or  cutting  plane  methods,  and  the  obtained  solutions  are  valid  lower  bounds  for 
the  QAP. 

In  terms  of  quality  the  bounds  obtained  in  this  way  are  competitive  with  the 
best  existing  lower  bounds  for  the  QAP.  For  many  test  instances  from  QAPLIB, 
such  as  some  instances  of  Hadley,  Roucairol,  Nugent  et  al.  and  Taillard,  they  are 
the  best  existing  bounds.  However,  due  to  prohibitively  high  computation  time 
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requirements,  the  use  of  such  approaches  as  basic  bounding  procedures  within 
branch  and  bound  algorithms  is  up  to  now  not  feasible. 

We  refer  to  [107,  110]  for  a detailed  description  of  SDP  approaches  to  the  QAP 
and  illustrate  the  idea  by  describing  just  one  semidefinite  programming  relaxation 
for  the  QAP. 

The  set  of  n x n permutation  matrices  X„  is  the  intersection  of  the  set  of  n x n 
0-1  matrices,  denoted  by  and  the  set  of  n x n matrices  with  row  and  column 
sums  equal  to  1.  Moreover,  X„  is  also  the  intersection  of  with  the  set  of  n x n 
orthogonal  matrices,  denoted  by  On-  Hence 


n = Zr.  n o 


n 


n 


Recall  that 


On  = {X  e 1 XX^  = X^X  = /}  and 


£n  = {X  e IR”^” 


Xu  = X'^u  = u} 


5 


where  I is  the  n x n identity  matrix  and  u is  the  n-dimensional  vector  of  all 
ones.  Then,  the  trace  formulation  of  the  QAP  (2.2.4)  with  the  additional  linear 
term 

n n 

— 2 ^ ^ ^ ^ bijXij , 
i=l  j=l 
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can  be  represented  equivalently  as  follows: 


QAPe  : min  tr{FXDX^  - 2BX^) 
s.t.  XX'^  = X'^X  = I, 
Xu  = X u = u, 

xfj  — Xij  = 0. 


In  order  to  obtain  a semidefinite  relaxation  for  the  QAP  from  the  formulation 
QAPs  above,  we  introduce  first  an  n^-dimensional  vector  vec{X).  vec{X)  is  ob- 
tained as  a column-wise  ordering  of  the  entries  of  matrix  X.  Then  the  vector 
vec{X)  is  lifted  into  the  space  of  (n^  -I-  1)  x (iP  -\-  1)  matrices  by  introducing  a 
matrix  Yx, 


vec{X)^ 

vec(X)vec(X)^  ^ 


Thus,  Yx  has  some  entry  xq  in  the  left-upper  corner  followed  by  the  vector  vec{X) 
in  its  first  row  (column).  The  remaining  terms  are  those  of  the  matrix  vec{X)vec{X)'^ 
sitting  on  the  right  lower  nf  x block  of  Yx-  Secondly,  the  coefficients  of  the 
problem  are  collected  in  an  (n^  -|- 1)  x -I- 1)  matrix  K given  as 


—vec{B)^ 
D®F  , 


where  the  operator  vec  is  defined  as  above  and  F is  the  Kronecker  product  of 
D and  F. 

It  is  easy  to  see  that  with  these  notations  the  objective  function  of  QAPs  equals 
tr{KYx).  By  setting  yoo  :=  xq  = 1 as  done  in  Zhao  et  al.  [110],  one  obtains  two 
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additional  constraints  to  be  fulfilled  by  the  matrix  Yx:  Vx  is  positive  semidefinite 
and  matrix  Yx  is  a rank-one  matrix.  Whereas  the  semidefiniteness  and  the  equality 
yoo  = 1 can  be  immediately  included  in  an  SDP  relaxation,  the  rank-one  condition 
is  hard  to  handle  and  is  discarded  in  an  SDP  relaxation.  In  order  to  assure  that 
the  rank-one  positive  semidefinite  matrix  Yx  is  obtained  by  an  n x n permutation 
matrix  as  described  above,  other  constraints  should  be  imposed  to  Yx.  Such 
conditions  can  be  formulated  as  valid  constraints  of  an  SDP  formulation  for  the 
QAP  by  means  of  some  new  operators,  acting  on  matrices  or  vectors  as  introduced 
below. 

diag{A)  produces  a vector  containing  the  diagonal  entries  of  matrix  A in  their 
natural  order,  that  is,  from  top-left  to  bottom-right.  The  adjoint  operator  Diag 
acts  on  a vector  V and  produces  a square  matrix  Diag{V)  with  off-diagonal  entries 
equal  to  0 and  the  components  of  V on  the  main  diagonal.  Clearly,  for  an  n 
dimensional  vector  V,  Diag{V)  is  an  n x n matrix. 

arrow  acts  on  an  (n^  + 1)  x (n^  -|- 1)  matrix  Y and  produces  an  -f- 1 dimensional 
vector  arrowiY)  = dzay(F)  — (0,  y'o.unOi  where  (0,  y(o,i:n2))  is  an  n^-\-l  dimensional 
vector  with  first  entry  equal  to  0 and  other  entries  coinciding  with  the  entries  of 
Y lying  on  the  0-th  row  and  in  columns  between  1 and  n^,  in  their  natural  order 
The  adjoint  operator  Arrow  acts  on  an  4- 1 dimensional  vector  W and  produces 
an  {n?  -I- 1)  x {'n?  4- 1)  matrix  Arrow{W) 


Arrow{W)  = 


Wo 


\ 


1/2IF(i:„2)  Diag{Wi,n'i)  j 


where  IT(i;„2)  is  the  dimensional  vector  obtained  from  W by  removing  its  first 
entry  wo- 

^Note  here  that  the  rows  and  columns  of  an  {m?  -i-  1)  x (n^  + 1)  matrix  are  indexed  by 

0, 1, . . . , 


76 


Further,  we  are  going  to  consider  an  (n^  + 1)  x + i)  matrix  V as  composed 
of  its  first  row  F(o,.),  of  its  first  column  F(.,o),  and  of  submatrices  of  size  n x n 
each,  which  are  arranged  in  an  n x n array  of  n x n matrices  and  produce  its 
remaining  x block  (this  is  similar  to  the  structure  of  a Kronecker  product  of 
two  n X n matrices,  see  §2.2.5).  The  entry  yap,  1 < a,  /?  < n^,  will  be  also  denoted 
by  V{ij)(ki),  with  1 < i,j,  k,l  < n,  where  a = (z  - l)n  + j and  p = {k  - l)n  + 1. 
Hence,  y{ij)(ki)  is  the  element  with  coordinates  {j,  1)  within  the  n x n block  with 
coordinates  (i,k). 

With  these  formal  conventions  let  us  define  the  so-called  block-Q- diagonal  and 
off-O-diagonal  operators,  acting  on  an  (n^  -h  1)  x (n^  + 1)  matrix  Y,  and  denoted 
by  b^diag  and  o^diag,  respectively,  b^diagiy)  and  o^diagiY)  are  n x n matrices 
given  as  follows: 


b^diag{Y)  = '^Y^k,.){k,.), 

k=l 


o^diag{Y)  = 

k=l 


where,  for  1 < A;  < n,  Y(k,.)(k,.)  is  the  A:-th  n x n matrix  on  the  diagonal  of  the 
n X n array  of  matrices,  defined  as  described  above.  Analogously,  Y(^,^k),(.,k)  is  an 
n X n matrix  consisting  of  the  diagonal  elements  sitting  on  the  position  {k,  k)  of 
the  n X n matrices  (n^  matrices  altogether)  which  form  the  x lower  right 
block  of  matrix  Y.  The  corresponding  adjoint  operators  B^Diag  and  O^Diag  act 
on  an  n X n matrix  S and  produce  (n^  -I- 1)  x (n^  -l- 1)  matrices  as  follows: 


B^Diag 


y 0 7®  5 J 


O^Diag 


^ 0 0 ^ 
y 0 S ® I t 


Finally,  let  us  denote  by  cq  the  n^-f  1 dimensional  unit  vector  with  first  component 
equal  to  1 and  all  other  components  equal  to  0,  and  let  R be  the  (n^  -f- 1)  x (n^  + 1) 
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matrix  given  by 


R = 

^ n 

—u^  0 

+ 

n 

—viF  0 ^ 

1 —u  0 u 

I0E  j 

. —u  0 u 

E0I 

where  E is  the  n x n matrix  of  all  ones. 

With  these  notations,  a semidefinite  relaxation  for  QAPs  is  given  as  follows 

min  tr{KY) 
s.t. 

b^diag{Y)  = I, 

{QAPro)  o^diag{Y)  = I, 

arrow{Y)  = eo, 
tr(RY)  = 0, 

r ^ 0. 

where  :<  is  the  so-called  Lowner  partial  order,  that  \s,  A < B if  and  only  if 
B — A y 0,  that  is  jB  — A is  positive  semidefinite. 

Zhao  et  al.  [110]  have  shown  that  an  equivalent  formulation  for  the  considered 
QAP  is  obtained  from  QAPro  by  imposing  one  additional  condition  on  the  matrix 
Y,  namely,  the  rank-one  condition. 

2.9  Exact  Solution  Methods 

An  exact  algorithm  for  a combinatorial  optimization  problem  provides  the 
global  optimal  solution  to  the  problem.  In  this  section  we  will  briefly  discuss  sev- 
eral exact  algorithms  that  have  been  used  for  solving  the  QAP.  More  specifically 
the  exact  algorithms  that  have  been  used  for  the  QAP  are  dynamic  programming. 
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cutting  plane  algorithms,  and  branch  and  bound  which  appears  to  be  the  most 
successful  one. 

2.9.1  Branch  and  Bound 

Branch  and  bound  algorithms  have  been  applied  successfully  to  many  combi- 
natorial optimization  problems,  and  they  appear  to  be  the  most  efficient  exact 
algorithms  for  solving  the  QAP.  Although  there  are  different  branch  and  bound 
techniques  we  will  try  to  describe  the  general  concept.  At  the  beginning  we  obtain 
a good  feasible  solution  to  the  problem  by  means  of  a heuristic,  and  we  treat  this 
solution  as  an  upper  bound.  The  problem  is  then  decomposed  into  a finite  number 
of  subproblems  and  a lower  bound  is  established  for  each.  Then  each  of  these  sub- 
problems is  decomposed  in  the  same  fashion,  and  by  doing  so  we  construct  a search 
tree.  However,  many  subproblems  are  ignored  using  a lower  bound,  a procedure 
called  pruning,  so  in  general  we  avoid  complete  enumeration  of  the  whole  feasible 
set.  For  the  QAP  there  are  three  types  of  branch  and  bound  algorithms,  namely: 

• Single  assignment  algorithms  (Gilmore  [81],  Lawler  [123]). 

• Pair  assignment  algorithms  (Gavett  and  Plyter  [79],  Land  [120],  Nugent  et 
al.  [147]). 

• Relative  positioning  algorithm  (Mirchandani  and  Obata  [139]). 

All  of  the  above  algorithms  work  by  iterative  constructing  an  optimal  permutation 
starting  from  an  empty  permutation.  The  single  assignment  algorithms  seem  to 
be  the  most  efficient  and  the  pair  assignment  algorithms  do  not  have  favorable 
computational  results.  Lower  bounds,  which  is  the  main  ingredient  for  an  efficient 
branch  and  bound  algorithm,  is  probably  one  of  the  most  extensively  studied 
aspects  of  the  QAP. 
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We  will  now  describe  a recent  branch  and  bound  algorithm  for  the  QAP,  that 
was  proposed  by  Pardalos  et  al.  [155].  In  the  description  that  follows  we  will 
consider  the  QAP  as  it  appears  in  the  facility  location  context,  where  and  input 
matrices  F,D  and  B are  given.  First  let  us  define  the  necessary  notation  used 
in  describing  the  branch  and  bound  algorithm.  A partial  permutation  for  the  set 


of  integers  J\f  = {1,2,...  , n}  is  denoted  by  pk  := 


1 


k 


\ 


\Pk{l)  Pk{2)  •••  Pk{k)  J 

where  k < n.  From  now  we  will  write  pk  = (pfc(l),pfc(2), . . . ,Pk{k))  for  short.  An 

assignment  of  a facility  i to  a location  j will  be  denoted  by  z jf,  while  if  i must 

never  be  assigned  to  j we  will  write  i j.  Note  that  Pk  is  essentially  a partial 

assignment  of  facilities  to  locations.  If  we  want  to  add  an  extra  assignment  to  some 

Pk,  say  A:  + 1 -4  j,  we  will  write  pk+i  = PkUk  + 1 j,  thereby  Pk+i{i)  = Pk{i)  for 

z = 1, . . . ,k,  and  p^;+i(A;  + 1)  = j-  Given  some  pk  let  its  range  be  Qk  :=  {pk{i)  \ i = 

1, . . . ,k},  and  define  the  sets  of  not-permissible  assignments  to  be  Ek+i  :=  {j  € 

J^\Qk  I k + 1 tA  j}.  For  example  for  rz  = 5,  the  partial  permutation  p2  = (3, 1) 

denotes  the  assignments  1^3  and  2^1,  while  E3  can  be  one  of  following  sets, 

{2},  {4}{5},  {2, 5},  {2, 4}  and  {4,5}.  Assuming  that  we  set  E3  = {2,5},  the  only 

possible  assignment  now  is  3 -4  4,  thereby  ps  = p2  U 3 ^ 4 = (3, 1, 4).  Given  a 

QAP  instance,  a pair  of  pk  and  Ek+i  completely  defines  a subproblem  Pi  as 


Pi  : min 

pen^v 

s.t. 


n n n 

i j i 

p{i)=Pk{i),  i = l,...,k, 


p{k  + 1)  ^ Ek+\- 


(2.63) 


The  original  problem  Pq  is  obtained  for  an  empty  partial  permutation  po  and 
El  = 0.  For  each  Pi  a lower  bound  g{Pi)  can  be  computed,  using  any  of  the 
described  methods  in  §2.8,  and  let  the  optimal  solution  to  Pi  be  denoted  by  f{Pi)- 
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In  the  branch  and  bound  algorithm,  a forest  of  n binary  trees  is  constructed,  where 
each  node  of  the  tree  corresponds  to  a partial  subproblem  Pj.  The  branching 
process  is  as  follows.  Given  a node  Pj  (i.e.  a subproblem)  defined  by  some  pk  and 
Pfc+i,  two  descendant  nodes  are  created,  the  left  child  P/  and  the  right  child  P[. 
For  P-  we  set  =pkUk  + l-^j  for  some  j ^ Ek  and  = 0,  while  for  P[ 
we  set  pI  = Pk  and  U j.  A node  which  has  pk  with  k = n — 1 cannot 

decomposed  further,  and  it  is  called  a terminal  node.  Immediately  we  can  identify 
the  following  properties 

• g{Pi)  < f{Pi)  for  any  node  Pj, 

• g{Pi)  = f{Pi)  if  Pi  is  a terminal  node, 

• g{Pj)  > 9(Pi)  it  Pi  has  descended  from  Pj. 


A node  defined  by  some  Pk  and  Ek+i  will  have  two  terminal  nodes  as  children 
if  A:  = n — 2.  Moreover,  for  any  node  |Pa:+i|  + k < n — 1,  while  if  equality 
holds  then  there  is  only  one  j ^ Pfc+i  and  only  one  left  child  is  generated  with 
Pk+i  = Pfc  U A:  + 1 -4  ; and  Ek+2  = 0- 

The  branch  and  bound  algorithm  in  Pardalos  et  al.  [155]  starts  by  computing 
an  upper  bound  solution  to  the  original  subproblem  Pq  by  means  of  a heuristic. 
Let  the  corresponding  permutation  be  p = (p(l),p(2), . . . ,p{n)).  Note  that  during 
the  process  of  the  algorithm  the  upper  bound  is  continuously  updated  whenever  a 
better  feasible  solution  is  found.  Then  n nodes  are  created,  where  for  each  Pj  for 
z = 1, . . . , n,  we  set  Pi  = po  U 1 p{i)-  E^  — 0,  and  g{Pi)  = 0.  Then  the  following 

steps  are  performed  at  each  iteration 


1.  Selection:  Here  we  choose  which  node  to  examine  next,  and  we  choose  the 


node  with  the  maximum  g{Pi). 
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2.  Branching:  Given  the  chosen  node  Pi  from  step  1,  we  create  two  new  nodes 
Pj  and  P[,  based  on  the  branching  scheme  described  previously.  We  set 
9{P[)  = 9{Pi)  and  we  compute  g{Pl). 


3.  Elimination:  If  g{Pl)  is  less  than  or  equal  to  the  current  upper  bound,  then 
the  node  Pj  is  pruned,  that  is,  marked  not  to  be  considered  in  step  1 in  the 
future. 


4.  Termination:  The  algorithm  stops  if,  and  only  if,  there  are  no  more  nodes  to 
be  considered  in  step  1. 


The  authors  in  Pardalos  et  al.  [155]  applied  the  above  described  branch  and  bound 
algorithm  for  the  QAP  in  conjunction  with  the  variance  reduction  lower  bounds 
described  in  §2.8.2,  and  solved  previously  unsolved  QAP  instances  of  dimension 
n = 16  and  n = 20. 


Example  2.4  In  this  example  we  will  apply  the  branch  and  bound  algorithm  de- 
scribed in  this  section  to  the  QAP  instance  given  in  example  (2.2).  We  will  use 
the  Gilmore- Lawler  lower  bound  described  in  ^2.8.1.  Recall  that  the  input  matrices 
are 
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and  g{Po)  = 215  as  computed  in  the  example  (2.2).  Before  we  proceed  with  the 
algorithm,  we  have  to  describe  how  to  compute  the  Gilmore-Lawler  lower  bound 
for  a given  subproblem  Pi  which  is  defined  by  some  partial  permutation  Pk  (note 
that,  as  dictated  by  step  2 of  the  algorithm,  we  only  compute  lower  bounds  for 
the  left  children  which  have  Ek+i  = 0j.  For  each  i > k -\-  1 define  the  vector 
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/(t,.)  :=  {fik+i,  ■ ■ ■ , fin)  where  the  element  fa  has  been  removed.  Also  for  each 
j € Af\Qk  define  the  vector  :=  {dji^,. . . ,dji^_jfy  where  It  G Af\Qk  for  t = 
1, . . . ,n  — k,  and  the  element  djj  has  been  removed.  Therefore  we  have  n — k 
vectors  f{i,.)id(^j^,)  G IR  ~ for  such  defined  i and  j.  Now  construct  the  matrix 
L = {lij)  G as  follows 

k 

~ {/(j,  )’ ^(jv))  y ] {fit^jPkit)  A-  ftidpk{t)j)  • 

t=l 


Then  the  Gilmore- Lawler  lower  bound  for  subproblem  Pi  defined  by  the  partial 
permutation  pk  is 


k k 


9(Pi)  = yZyZ kdp>^{i)Pk(3)  + trLX'^, 

1=1  jj=l 


where  the  first  term  is  the  constant  cost  associated  with  the  partial  permutation 
Pk  and  the  second  is  an  LAP  with  cost  matrix  L.  Initially  we  have  to  compute 
an  upper  bound,  and  we  do  that  by  choosing  a random  permutation  p — (2, 4, 1,  3) 
which  gives  an  upper  bound  of  237.  Constructing  the  first  n nodes  and  applying 
steps  1 through  4 of  the  branch  and  bound  algorithm,  without  updating  the  upper 
bound  whenever  a feasible  solution  is  found,  the  whole  search  tree  can  be  seen  in 
Figure  2.1. 

We  can  see  that  two  optimal  solutions  exist  for  this  problem,  with  permutations 
= (3, 2, 4, 1)  and  p^  = (1, 4, 2, 3)  corresponding  to  nodes  31  and  25  of  the  search 
tree  respectively,  with  an  optimal  value  of  217.  Note  that  in  step  1 of  the  algorithm, 
we  choose  the  node  with  the  maximum  lower  bound  to  examine  first,  therefore  in 
the  case  were  more  than  one  nodes  have  equal  lower  bounds,  a node  among  them  is 
chosen  randomly.  In  the  example  we  chosen  the  nodes  corresponding  to  the  same 
binary  tree.  Also  note  that  at  node  15  we  have  a feasible  solution  of  231  < 237  the 
current  upper  bound,  however  we  did  not  update  the  upper  bound.  If  we  were  to  do 
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Upper  Bound  = 237  with  p = (2, 4, 1,3) 


Optimal 


_L 

i 


Prune  node 


Feasible  solution  node 


Figure  2.1:  Branch-Bound  search  tree  for  example  (2.4) 
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50,  nodes  17,21  and  34  would  have  been  pruned  and  we  would  not  decompose  them 
further. 

2.9.2  Traditional  Cutting  Plane  Methods 

Traditional  cutting  plane  algorithms  for  the  QAP  have  been  developed  by  a 
different  authors,  Bazaraa  and  Sherali  [19,  20],  Balas  and  Mazzola  [11,  12,  13],  and 
Kaufmann  and  Broeckx  [112].  These  algorithms  make  use  of  mixed  integer  linear 
programming  (MILP)  formulations  for  the  QAP  which  are  suitable  for  Benders’ 
decomposition.  In  the  vein  of  Benders,  the  MILP  formulation  is  decomposed  into 
a master  problem  and  a subproblem,  called  also  slave  problem,  where  the  master 
problem  contains  the  original  assignment  variables  and  constraints.  For  a fixed 
assignment  the  slave  problem  is  usually  a linear  program  and  hence,  solvable  in 
polynomial  time.  The  master  problem  is  a linear  program  formulated  in  terms  of 
the  original  assignment  variables  and  of  the  dual  variables  of  the  slave  problem, 
and  is  solvable  in  polynomial  time  for  fixed  values  of  those  dual  variables.  The 
algorithms  work  typically  as  follows.  First,  a heuristic  is  applied  to  generate  a 
starting  assignment.  Then  the  slave  problem  is  solved  for  fixed  values  of  the 
assignment  variables  implied  by  that  assignment,  and  optimal  values  of  the  primal 
and  dual  variables  are  computed.  If  the  dual  solution  of  the  slave  problem  satisfies 
all  constraints  of  the  master  problem,  we  have  an  optimal  solution  for  the  original 
MILP  formulation  of  the  QAP.  Otherwise,  at  least  one  of  the  constraints  of  the 
master  problem  is  violated.  In  this  case,  the  master  problem  is  solved  with  fixed 
values  for  the  dual  variables  of  the  slave  problem  and  the  obtained  solution  is  given 
as  input  to  the  slave  problem.  The  procedure  is  then  repeated  until  the  solution 
of  the  slave  problem  fulfills  all  constraints  of  the  master  problem. 

Clearly  any  solution  of  the  master  problem  obtained  by  fixing  the  dual  variables 
of  the  slave  problem  to  some  feasible  values,  is  a lower  bound  for  the  considered 
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QAP.  On  the  other  side,  the  objective  function  value  of  the  QAP  corresponding 
to  any  feasible  setting  of  the  assignment  variables  is  an  upper  bound.  The  algo- 
rithm terminates  when  the  lower  and  the  upper  bounds  coincide.  Generally,  the 
time  needed  for  the  upper  and  the  lower  bounds  to  converge  to  a common  value 
is  too  large,  and  hence  these  methods  may  solve  to  optimality  only  very  small 
QAPs.  However,  heuristics  derived  from  cutting  plane  approaches  produce  good 
suboptimal  solutions  in  early  stages  of  the  search,  see  for  example,  Burkard  and 
Bonniger  [28]  and  Bazaraa  and  Sherali  [20]. 

2.9.3  Polyhedral  Cutting  Planes 

Similarly  to  traditional  cutting  plane  methods  also  polyhedral  cutting  planes 
or  branch  and  cut  algorithms'^  make  use  of  an  LP  or  MILP  relaxation  of  the  com- 
binatorial optimization  problem  to  be  solved,  in  our  case  the  QAP.  Additionally, 
polyhedral  cutting  plane  methods  make  use  of  a class  of  (nontrivial)  valid  or  facet 
defining  inequalities  known  to  be  fulfilled  by  all  feasible  solutions  of  the  original 
problem.  If  the  solution  of  the  relaxation  is  feasible  for  the  original  problem,  we 
are  done.  Otherwise,  some  of  the  above  mentioned  valid  inequalities  are  probably 
violated.  In  this  case  a “cut”  is  performed,  that  is,  one  or  more  of  the  violated 
inequalities  are  added  to  the  LP  or  MILP  relaxation  of  our  problem.  The  latter 
is  resolved  and  the  whole  process  is  repeated.  In  the  case  that  none  of  the  valid 
inequalities  is  violated,  but  some  integrality  constraint  is  violated,  the  algorithm 
performs  a branching  step  by  fixing  (feasible)  integer  values  for  the  corresponding 
variable.  The  branching  steps  produce  the  search  tree  like  in  branch  and  bound 
algorithms.  Each  node  of  this  tree  is  processed  as  described  above  by  performing 
cuts  and  then  by  branching  it,  if  necessary.  Clearly,  related  elements  of  branch  and 

bound  algorithms  like  upper  bounds,  selection  and  branching  rules  play  a role  in 
^This  term  was  originally  used  by  Padberg  and  Rinaldi  [150]. 
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branch  and  cut  algorithms.  Hence,  such  an  approach  combines  elements  of  cutting 
plane  and  branch  and  bound  methods. 

The  main  advantage  of  polyhedral  cutting  plane  algorithms  with  respect  to 
traditional  cutting  planes  relies  on  the  use  of  cuts  which  are  valid  for  the  whole 
polytope  of  the  feasible  solutions,  and  possibly  facet  defining.  Traditional  cutting 
planes  instead  rely  frequently  on  cuts  which  are  not  valid  for  the  whole  polytope 
of  the  feasible  solutions.  In  this  case  the  whole  computation  has  to  be  done  from 
scratch  for  different  variable  fixings.  This  requires  additional  running  time  and 
additional  amounts  of  memory.  Another  and  not  less  important  drawback  of  tra- 
ditional cutting  plane  algorithms  is  due  to  the  “weakness”  of  the  cuts  they  involve. 
In  contrast  with  cuts  produced  by  facet  defining  inequalities,  the  weak  cuts  cannot 
avoid  the  slow  convergence. 

As  we  saw  in  §2.5  some  properties  and  few  facet  defining  inequalities  of  the 
QAP  polytope  are  already  known.  But  still  polyhedral  cutting  plane  methods 
for  the  QAP  are  not  yet  backed  by  a strong  theory.  However,  some  efforts  to 
design  branch  and  cut  algorithms  for  the  QAP  have  been  made  by  Padberg  and 
Rijal  [149]  and  Kaibel  [106].  Padberg  and  Rijal  [149]  have  tested  their  algorithm 
on  sparse  QAP  instances  from  QAPLIB.  The  numerical  results  are  encouraging, 
although  the  developed  software  is  of  preliminary  nature,  as  claimed  by  the  au- 
thors. Kaibel  [106]  has  used  branch  and  cut  to  compute  lower  bounds  for  QAP 
instances  from  QAPLIB.  His  results  are  promising  especially  in  the  case  where  box 
inequalities  are  involved. 

2.10  Heuristics 

Although  substantial  improvements  have  been  done  in  the  development  of  ex- 
act algorithms  for  the  QAP,  problems  of  dimension  n > 20  are  still  not  practical  to 
solve  because  of  very  high  computer  time  requirements.  This  makes  the  develop- 
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merit  of  heuristics  indispensable  as  algorithms  which  provide  good  quality  solutions 
in  a reasonable  time.  Much  research  has  been  devoted  to  the  development  of  such 
approaches.  We  distinguish  the  following  types  of  heuristic  algorithms: 

• Construction  methods  (CM) 

• Limited  enumeration  methods  (LEM) 

• Improvement  methods  (IM) 

• Tabu  search  (TS) 

• Simulated  annealing  (SA) 

• Genetic  algorithms  (GA) 

• Greedy  randomized  adaptive  search  procedures  (GRASP) 

• Ant  systems  (AS) 

2.10.1  Construction  Methods 

Construction  methods  were  introduced  by  Gilmore  [81].  They  are  iterative 
approaches  which  usually  start  with  an  empty  permutation,  and  iteratively  com- 
plete a partial  permutation  into  a solution  of  the  QAP  by  assigning  some  facility 
which  has  not  been  assigned  yet  to  some  free  location.  The  algorithm  is  presented 
in  pseudocode  in  Figure  2.2.  Here  po,Pi)---  ,Pn-i  are  partial  permutations,  and 
heur(i)  is  some  heuristic  procedure  that  assigns  facility  i to  some  location  j,  and 
returns  j.  P is  the  set  of  already  assigned  pairs  of  facilities  to  locations.  The  pro- 
cedure update  constructs  a permutation  pi  by  adding  the  assignment  (i,j)  to  pj_i. 
The  heuristic  heur(z)  employed  by  update  could  be  any  heuristic  which  chooses  a 
location  j for  facility  i,  (i,j)  ^ P,  in  a greedy  fashion  or  by  applying  local  search. 
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procedure  constructionfpo,  F) 

1 

p = {}; 

2 

do  f = 1 

j . . . ,77/  — 1 — y 

3 

if(i. 

i)  ^ r 

4 

j =heur(i); 

5 

update(pi,(i,j)); 

6 

r = FU 

7 

fi; 

8 

p = 

Pi"', 

9 

od; 

10 

return(p) 

end  construction; 

Figure  2.2:  Pseudo-code  for  a construction  method 

One  of  the  oldest  heuristics  used  in  practice,  the  CRAFT  heuristic  developed  by 
Buffa,  Armour  and  Vollmann  [24],  is  a construction  method.  Another  construction 
method  which  yields  good  results  has  been  proposed  by  Miiller-Merbach  [143]. 


2.10.2  Limited  Enumeration  Methods 

Limited  enumeration  methods  rely  on  the  observation  that  often  enumeration 
methods  (e.g.  branch  and  bound  algorithms)  find  good  solutions  in  early  stages 
of  the  search,  and  employ  then  a lot  of  time  to  marginally  improve  that  solution 
or  prove  its  optimality.  This  behavior  of  enumeration  methods  suggests  a way  to 
save  time  in  the  case  that  we  are  interested  in  a good  but  not  necessarily  optimal 
solution:  impose  some  limit  to  the  enumeration  process.  This  limit  could  be  a 
time  limit,  or  a limit  on  the  number  of  iterations  the  algorithm  may  perform. 

Another  strategy  which  serves  the  same  goal  is  to  manipulate  the  lower  bound. 
This  can  be  done  by  increasing  the  lower  bound  if  no  improvement  in  the  solution 
is  achieved  during  a large  number  of  iterations,  and  would  yield  deeper  cuts  in  the 
search  tree  to  speed  up  the  process.  Clearly,  such  an  approach  may  cut  off  the 
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optimal  solution  and  hence  should  be  used  carefully,  possibly  in  conjunction  with 
certain  heuristics  that  perform  elaborate  searches  in  the  feasible  space. 

2.10.3  Improvement  Methods 

These  methods  belong  to  the  larger  class  of  local  search  algorithms.  A local 

# 

search  procedure  starts  with  an  initial  feasible  solution  and  iteratively  tries  to  im- 
prove the  current  solution.  This  is  done  by  substituting  the  latter  with  a (better) 
feasible  solution  from  its  neighborhood.  This  iterative  step  is  repeated  until  no 
further  improvement  can  be  found.  Improvement  methods  are  local  search  algo- 
rithm which  allow  only  improvements  of  the  current  solution  in  each  iteration.  For 
a comprehensive  discussion  of  theoretical  and  practical  aspects  of  local  search  in 
combinatorial  optimization  the  reader  is  referred  to  the  book  edited  by  Aarts  and 
Lenstra  [2]. 

Basic  ingredients  of  improvement  methods  (and  of  local  search  in  general)  are 
the  neighborhood  and  the  order  in  which  the  neighborhood  is  scanned.  Frequently 
used  neighborhoods  for  QAPs  are  the  pair-exchange  neighborhood  and  the  cyclic 
triple- exchange  neighborhood.  In  the  case  of  pair-exchanges  the  neighborhood  of 
a given  solution  (permutation)  consists  of  all  permutations  which  can  be  obtained 
from  the  given  one  by  applying  a transposition  to  it.  In  this  case,  scanning  the 
whole  neighborhood,  that  is,  computing  the  objective  function  values  for  all  neigh- 
bors of  a given  permutation,  takes  O(n^)  time.  (The  size  of  the  neighborhood 
is  (”),  and  it  takes  0{n)  steps  to  compute  the  difference  of  the  objective  func- 
tion values  of  a permutation  p and  a permutation  p'  in  the  neighborhood  of  p.) 
If  the  neighborhood  of  p is  already  scanned  and  p'  is  a neighbor  of  p,  then  the 
neighborhood  of  p'  can  be  scanned  in  O(n^),  see  Frieze  et  al.  [76]. 

In  the  case  of  cyclic  triple-exchanges,  the  neighborhood  of  a solution  (permu- 
tation) p consists  of  all  permutations  obtained  from  p by  a cyclic  exchange  of  some 
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triple  of  indices.  The  size  of  this  neighborhood  is  (^((g)).  Cyclic  triple-exchanges 
do  not  really  lead  to  better  results  when  compared  with  pair-exchanges. 

Another  important  ingredient  of  improvement  methods  is  the  order  in  which 
the  neighborhood  is  scanned.  This  order  can  be  either  fixed  previously  or  chosen 
at  random.  Given  a neighborhood  structure  and  a scanning  order,  a rule  for  the 
update  of  the  current  solution  (from  the  current  iteration  to  the  subsequent  one) 
should  be  chosen.  The  following  update  rules  are  frequently  used: 

• First  improvement 

• Best  improvement 

• Heider’s  rule  [97] 

In  the  case  of  first  improvement  the  current  solution  is  updated  as  soon  as  the 
first  improving  neighbor  solution  is  found.  Best  improvement  scans  the  whole 
neighborhood  and  chooses  the  best  improving  neighbor  solution  (if  such  a solution 
exists  at  all).  Heider’s  rule  starts  by  scanning  the  neighborhood  of  the  initial 
solution  in  a prespecified  cyclic  order.  The  current  solution  is  updated  as  soon  as 
an  improving  neighbor  solution  is  found.  The  scanning  of  the  neighborhood  of  the 
new  solution  starts  there  where  the  scanning  of  the  previous  one  was  interrupted 
(in  the  prespecified  cyclic  order). 

In  order  to  get  better  results,  improvement  methods  and  local  search  algorithms 
in  general  are  performed  several  times  starting  with  different  initial  solutions. 

2.10.4  Tabu  Search 

Tabu  search  is  a local  search  method  introduced  by  Glover  [84,  85]  as  a tech- 
nique to  overcome  local  optimality.  One  way  to  overcome  local  optimality  would 
be  to  allow  also  the  deterioration  of  the  current  solution  when  moving  from  one 
iteration  to  the  subsequent  one,  in  contrast  to  improvement  methods.  In  the  case 
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of  tabu  search  the  basic  idea  is  to  “remember”  which  solutions  have  been  visited  in 
the  course  of  the  algorithm,  in  order  to  derive  the  promising  directions  for  further 
search.  Thus,  the  memory  and  not  only  the  local  investigation  of  the  neighborhood 
of  the  current  solution  drives  the  search.  The  reader  is  referred  to  the  book  edited 
by  Glover,  Laguna,  Taillard,  and  De  Werra  [83]  for  a comprehensive  introduction 
to  tabu  search  algorithms. 

The  main  ingredients  of  tabu  search  are  the  neighborhood  structure,  the  moves, 
the  tabu  list  and  the  aspiration  criterion.  A move  is  an  operation  which,  when 
applied  to  a certain  solution  p,  generates  a neighbor  p'  of  it.  In  the  case  of  QAPs 
the  neighborhood  is  the  pair-exchange  neighborhood  and  the  moves  are  usually 
transpositions.  A tabu  list  is  a list  of  forbidden  or  tabu  moves,  that  is,  moves 
which  are  not  allowed  to  be  applied  to  the  current  solution.  The  tabu  status  of 
the  moves  changes  along  with  the  search  and  the  tabu  list  is  updated  during  the 
search.  An  aspiration  criterion  is  a condition  which,  when  fulfilled  by  a tabu  move, 
cancels  its  tabu  status. 

A generic  tabu  search  procedure  starts  with  an  initial  feasible  solution  S and 
selects  a best-quality  solution  among  (a  part  of)  the  neighbors  of  S obtained  by 
non-tabu  moves.  Note  that  this  neighboring  solution  does  not  necessarily  improve 
the  value  of  the  objective  function.  Then  the  current  solution  is  updated,  that  is, 
it  is  substituted  by  the  selected  solution.  Obviously,  this  procedure  can  cycle,  that 
is,  visit  some  solution  more  than  once.  In  an  effort  to  avoid  this  phenomenon  a 
tabu  criterion  is  introduced  in  order  to  identify  moves  which  are  expected  to  lead 
to  cycles.  Such  moves  are  then  declared  tabu  and  are  added  to  the  tabu  list.  As, 
however,  forbidding  certain  moves  could  prohibit  visiting  “interesting”  solutions, 
an  aspiration  criterion  distinguishes  the  potentially  interesting  moves  among  the 
forbidden  ones.  The  search  stops  when  a stop  criterion  (running  time  limit,  limited 
number  of  iterations)  is  fulfilled. 
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There  is  a lot  of  freedom  in  the  implementation  of  different  elements  of  a tabu 
search  algorithms,  that  is,  the  tabu  list  (length  and  maintenance),  the  aspiration 
criterion,  the  tabu  criterion.  The  performance  of  tabu  search  algorithms  depends 
very  much  on  the  implementation  chosen  for  its  basic  ingredients,  and  there  is  no 
general  agreement  about  the  best  implementation  of  any  of  those. 

Different  implementations  of  tabu  search  have  been  proposed  for  the  QAP, 
for  example,  a tabu  search  with  fixed  tabu  list  (Skorin-Kapov  [177]),  the  robust 
tabu  search  (Taillard  [182]),  where  the  size  of  the  tabu  list  is  randomly  chosen 
between  a maximum  and  a minimum  value,  and  the  reactive  tabu  search  (Battiti 
and  Tecchiolli  [18])  which  involves  a mechanism  for  adopting  the  size  of  the  tabu 
list.  Reactive  tabu  search  aims  at  improving  the  robustness  of  the  algorithm.  The 
algorithm  notices  when  a cycle  occurs,  and  increases  the  tabu  list  size  according 
to  the  length  of  the  detected  cycle.  The  numerical  results  show  that  generally  the 
reactive  tabu  search  outperforms  other  tabu  search  algorithms  for  the  QAP  (see 
[18)). 

More  recently,  also  parallel  implementations  of  tabu  search  have  been  proposed, 
see  for  example,  Chakrapani  and  Skorin-Kapov  [47].  Tabu  search  algorithms  allow 
a natural  parallel  implementation  by  dividing  the  burden  of  the  search  in  the 
neighborhood  among  several  processors. 

2.10.5  Simulated  Annealing 

Simulated  annealing  is  a local  search  approach  which  exploits  the  analogy  be- 
tween combinatorial  optimization  problems  and  problems  from  statistical  mechan- 
ics. Kirkpatrick,  Gelatt  and  Vecchi  [114]  and  Cerny  [46]  were  among  the  first 
authors  who  recognized  this  analogy,  and  showed  how  the  Metropolis  algorithm 
(Metropolis,  Rosenbluth,  Rosenbluth,  Teller,  and  Teller  [137])  used  to  simulate  the 
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behavior  of  a physical  many-particle  system  can  be  applied  as  a heuristic  for  the 
traveling  salesman  problem. 

The  analogy  between  a combinatorial  optimization  problem  and  a many- 
particle  physical  system  basically  relies  on  two  facts: 

• Feasible  solutions  of  the  combinatorial  optimization  problem  correspond  to 
states  of  the  physical  system. 

• The  objective  function  values  corresponds  to  the  energy  of  the  the  states  of 
the  physical  system. 

In  condensed  matter  physics  annealing  is  known  as  a cooling  process  which  pro- 
duces low  energy  thermal  equilibrium  states  of  a solid  in  a heat  bath.  The  aim  is 
to  reach  the  so-called  ground  state  which  is  characterized  by  a minimum  of  energy. 

Burkard  and  Rendl  [38]  showed  that  a simulated  cooling  process  yields  a gen- 
eral heuristic  which  can  be  applied  to  any  combinatorial  optimization  problem, 
as  soon  as  a neighborhood  structure  has  been  introduced  in  the  set  of  its  feasible 
solutions.  In  particular  Burkard  et  al.  applied  simulated  annealing  to  the  QAP. 
Other  simulated  annealing  (SA)  algorithms  for  the  QAP  have  been  proposed  by 
different  authors,  see  for  example,  Wilhelm  and  Ward  [186]  and  Connolly  [52]. 
All  these  algorithms  employ  the  pair-exchange  neighborhood.  They  differ  on  the 
way  the  cooling  process  or  the  thermal  equilibrium  is  implemented.  The  numeri- 
cal experiments  show  that  the  performance  of  SA  algorithms  strongly  depends  on 
the  values  of  the  control  parameters,  and  especially  on  the  choice  of  the  cooling 
schedule. 

Simulated  annealing  (SA)  can  be  modeled  mathematically  by  an  inhomoge- 
neous ergodic  Markov  chain,  and  this  model  has  been  used  for  the  probabilistic 
analysis  of  the  convergence  of  simulated  annealing  algorithms.  Under  natural  con- 
ditions on  the  involved  neighborhood  structure  and  non  very  restrictive  conditions 
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on  the  slowness  of  the  cooling  process  it  can  be  shown  that  SA  asymptotically 
converges  to  an  optimal  solution  of  the  considered  problem.  The  investigation  of 
the  speed  of  this  convergence  remains  an  (apparently  difficult)  open  problem.  For 
a detailed  discussion  on  the  convergence  and  other  theoretical  aspects  of  simulated 
annealing  the  reader  is  referred  to  the  books  by  Aarts  and  Korst  [1]  and  Laarhoven 
and  Aarts  [185]. 

2.10.6  Genetic  Algorithms 

The  so-called  genetic  algorithms  (GA)  are  a nature  inspired  approach  for  com- 
binatorial optimization  problems.  The  basic  idea  is  to  adapt  the  evolutionary 
mechanisms  acting  in  the  selection  process  in  nature  to  combinatorial  optimiza- 
tion problems.  The  first  genetic  algorithm  for  optimization  problems  was  proposed 
by  Holland  [98]  in  1975. 

A genetic  algorithm  starts  with  a set  of  initial  feasible  solutions  (generated 
randomly  or  by  using  some  heuristic)  called  the  initial  population.  The  elements 
of  a population  are  usually  termed  “individuals” . The  algorithm  selects  a number 
of  pairs  of  individuals  or  parents  from  the  current  population  and  uses  so-called 
cross-over  rules  to  produce  some  feasible  solution  or  child  out  of  each  pair  of 
individuals.  Further,  a number  of  “bad”  solutions,  that  is,  solutions  yielding  to 
high  values  of  the  objective  function,  is  thrown  out  of  the  current  population.  This 
process  is  repeated  until  a stop  criterion,  that  is,  a time  limit,  a limit  on  the  number 
of  iterations,  a measure  of  convergence,  is  fulfilled.  In  the  course  of  the  algorithm, 
mutations  or  immigrations  are  applied  periodically  to  the  current  population  to 
improve  its  overall  quality  by  modifying  some  of  the  individuals  or  replacing  them 
by  better  ones,  respectively.  Often  local  optimization  tools  are  periodically  used 
within  GAs  resulting  in  so-called  hybrid  algorithms.  The  search  is  diversified  by 
means  of  so-called  tournaments.  A tournament  consists  of  applying  several  runs 
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of  a GA  starting  from  different  initial  populations  and  stopping  them  before  they 
converge.  A “better”  population  is  derived  as  a union  of  the  final  populations  of 
these  different  runs,  and  then  a new  run  of  the  GA  is  started  over  this  population. 
For  a good  coverage  of  theoretical  and  practical  issues  on  genetic  algorithms  the 
reader  is  referred  to  Davis  [56]  and  Goldberg  [87]. 

A number  of  authors  have  proposed  genetic  algorithms  for  the  QAP.  Standard 
algorithms  like  the  one  developed  by  Tate  and  Smith  [183],  have  difficulties  to 
generate  the  best  known  solutions  even  for  QAPs  of  small  or  moderate  size.  Hy- 
brid approaches,  such  as  combinations  of  GA  techniques  with  tabu  search  as  the 
one  developed  by  Fleurent  and  Ferland  [72]  seem  to  be  more  promising.  More  re- 
cently another  hybrid  algorithm,  the  so-called  greedy  genetic  algorithm  proposed 
by  Ahuja,  Orlin,  and  Tivari  [7]  produced  very  good  results  on  large  scale  QAPs 
from  QAPLIB. 

2.10.7  Greedy  Randomized  Adaptive  Search  Procedure 

The  greedy  randomized  adaptive  search  procedure  (GRASP)  was  introduced  by 
Feo  and  Resende  [69]  and  has  been  applied  successfully  to  different  hard  combina- 
torial optimization  problems  [68,  115,  116,  169]  and  among  them  to  the  QAP  [130, 
153]  and  the  BiQAP  [135].  The  reader  is  referred  to  [69]  for  a survey  and  tutorial 
on  GRASP. 

GRASP  is  a combination  of  greedy  elements  with  random  search  elements  in  a 
two  phase  heuristic.  It  consists  of  a construction  phase  and  a local  improvement 
phase.  In  the  construction  phase  good  solutions  from  the  available  feasible  space 
are  constructed,  whereas  in  the  local  improvement  phase  the  neighborhood  of  the 
solution  constructed  in  the  first  phase  is  searched  for  possible  improvements.  A 
pseudocode  of  GRASP  is  shown  in  Figure  2.3.  The  input  parameters  are  the  size 
RCLsize  of  the  restricted  candidate  list  (RCL),  a maximum  number  of  iterations. 
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and  a random  seed.  RCL  contains  the  candidates  upon  which  the  sampling  related 
to  the  construction  of  a solution  in  the  first  phase  will  be  performed. 

procedure  GRASP(RCLSize,MaxIter,RandomSeed) 

1 Inputinstance 0 ; 

2 do  A:  = 1, . . . , Maxiter  — > 

3 ConstructGreedyRandomizedSolut ion (RCLSize , RandomSeed) ; 

4 LocalSearch(BestSolutionFound); 

5 UpdateSolution(BestSolutionFound) ; 

6 od; 

7 return  BestSolutionFound 
end  GRASP; 


Figure  2.3:  Pseudo-code  for  a generic  GRASP 


2.10.8  Ant  Systems 

Ant  system  (AS)  are  recently  developed  heuristic  techniques  for  combinatorial 
optimization  which  try  to  imitate  the  behavior  of  an  ant  colony  in  search  for  food. 
Initially  the  ants  search  for  food  in  the  vicinity  of  their  nest  in  a random  way.  As 
soon  as  an  ant  finds  a source  of  food,  it  takes  some  food  from  the  source  and  carries 
it  back  to  the  nest.  During  this  trip  back  the  ant  leaves  a trail  of  a substance  called 
pheromone  on  the  ground.  The  pheromone  trail  serves  to  guide  the  future  search 
of  ants  towards  the  already  found  source  of  food.  The  intensity  of  the  pheromone 
on  the  trail  is  proportional  to  the  amount  of  food  found  in  the  source.  Thus,  the 
ways  to  rich  sources  of  food  visited  frequently  (by  a large  number  of  ants)  will  be 
indicated  by  stronger  pheromone  trails.  In  an  attempt  to  imitate  the  behavior  of 
ants  to  derive  algorithms  for  combinatorial  optimization  problems,  the  following 
analogies  can  be  exploited:  a)  the  area  searched  by  the  ants  resembles  the  set  of 
feasible  solutions,  b)  the  amount  of  food  at  food  sources  resembles  the  value  of  the 
objective  function,  and  c)  the  pheromone  trail  resembles  a component  of  adaptive 


memory. 
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AS  were  originally  introduced  by  Dorigo  [59]  and  Colorni,  Dorigo,  and 
Maniezzo  [50]  and  have  already  produced  good  results  for  well  known  problems 
like  the  traveling  salesman  problem  (TSP)  and  the  QAP  [51,  77]. 

In  the  case  of  the  QAP  the  pheromone  trail  which  is  also  the  key  element  of 
an  AS,  is  implemented  by  a matrix  T = (ry).  Tij  is  a measure  for  the  desirability 
of  locating  facility  i at  location  j in  the  solutions  generated  by  the  ants  (the 
algorithm).  To  illustrate  the  idea  we  briefly  describe  the  algorithm  of  Gambardella, 
Taillard  and  Dorigo  [77]. 

The  algorithm  is  iterative  and  constructs  a flxed  number,  say  m,  of  solutions  in 
each  iteration.  (This  number  is  a control  parameter  and  is  also  thought  as  number 
of  ants.)  In  the  first  iteration  these  solutions  are  generated  randomly,  whereas  in 
the  subsequent  iteration  they  are  updated  by  exploiting  the  information  contained 
in  the  pheromone  trail  matrix  T.  Initially  the  pheromone  trail  matrix  is  a constant 
matrix;  the  constant  is  inverse-proportional  to  the  best  value  of  the  objective 
function  found  so  far.  This  is  in  compliance  with  the  behavior  of  ants  whose 
search  directions  are  initially  chosen  at  random.  Let  us  denote  the  best  solution 
found  so  far  by  p*  and  its  corresponding  value  of  the  objective  function  by  f{p*)- 
In  the  further  iterations  the  entries  rjp.(j)  of  T are  increased  by  the  same  value 
which  is  proportional  to  f{p*)-  The  update  of  the  m solutions  in  each  iteration 
is  done  first  by  means  of  the  pheromone  trail  matrix,  and  then  by  applying  some 
improvement  method.  In  both  cases  the  update  consists  of  swapping  the  locations 
for  a sequence  of  facility  pairs.  First,  the  current  solution  is  updated  by  swapping 
the  locations  of  pairs  of  facilities  chosen  so  as  to  maximize  the  (normalized)  sum 
of  the  corresponding  pheromone  trail  entries.  Then,  the  solution  obtained  after 
this  update  is  improved  by  applying  some  improvement  methods,  such  as  first 
or  best  improvement  (see  also  §2.10.3).  As  soon  as  an  improvement  of  the  best 
known  solution  is  detected  an  intensification  component  “forces  the  ants”  to  further 
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explore  the  part  of  the  solution  space  where  the  improvement  was  found.  If  after 
a large  number  of  iterations  there  is  no  improvement  of  the  best  known  solution, 
a diversification  - which  is  basically  a new  random  start  - is  performed. 

Numerical  results  [51,  77]  show  that  ant  systems  are  competitive  heuristics  es- 
pecially for  real  life  instances  of  the  QAP  with  a few  very  good  solutions  clustered 
together.  For  randomly  generated  instances  which  have  many  good  solutions  dis- 
tributed somehow  uniformly  in  the  search  space,  AS  are  outperformed  by  other 
heuristics,  that  is,  genetic  algorithms  or  tabu  search  approaches. 

2.11  Polynomially  Solvable  Cases 

Since  the  QAP  is  NP-hard,  restricted  versions  which  can  be  solved  in  polyno- 
mial time  are  an  interesting  aspect  of  the  problem.  A basic  question  arising  with 
respect  to  polynomially  solvable  versions  is  the  identification  of  those  versions  and 
the  investigation  of  the  border  line  between  hard  and  easy  versions  of  the  prob- 
lem. There  are  two  ways  to  approach  this  topic:  first,  find  structural  conditions 
to  be  imposed  on  the  coefficient  matrices  of  the  QAP  so  as  to  obtain  polynomi- 
ally solvable  versions,  and  secondly,  investigate  other  combinatorial  optimization 
or  graph-theoretical  problems  which  can  be  formulated  as  QAPs,  and  embed  the 
polynomially  solvable  versions  of  the  former  into  special  cases  of  the  later.  These 
two  approaches  yield  two  groups  of  restricted  QAPs  which  are  briefly  reviewed  in 
this  section.  For  a detailed  information  on  this  topic  the  reader  is  referred  to  [45]. 

Most  of  the  restricted  versions  of  the  QAP  with  specially  structured  matrices 
involve  Monge  matrices  or  other  matrices  having  analogous  properties.  A matrix 
A = (ttij)  is  a Monge  matrix  iff  the  following  inequalities  are  fulfilled  for  each 
4-tuples  of  indices  i,j,  k,l,  i < k,  j < 1: 


99 


o-ij  + 0,kl  ^ 0.U  + a^j, 


(Monge  inequalities). 


A matrix  A = (0^)  is  an  Anti-Monge  matrix  iff  the  following  inequalities  are 
fulfilled  for  each  4-tuples  of  indices  i,j,  k,  I,  i < k,  j < 1: 


^ij  ^kl  ^ ^kj  ? 


(Anti-Monge  inequalities). 


a)  b) 

Figure  2.4:  The  sum  of  the  depicted  entries  taken  with  the  corresponding  signs 
must  be  nonnegative:  a)  Monge  inequality,  b)  Anti-Monge  inequality 

A simple  example  of  Monge  and  Anti-Monge  matrices  are  the  sum  matrices] 
the  entries  of  a sum  matrix  matrix  A = (ojj)  are  given  as  aij  = ai  + pj,  where  {ap 
and  (Pj)  are  the  generating  row  and  column  vector,  respectively.  A product  matrix 
A is  defined  in  an  analogous  way:  its  entries  are  given  as  aij  = aiPj,  where  {ap, 
(Pj)  are  the  generating  vectors.  If  the  row  generating  vector  (aj)  and  the  column 
generating  vectors  {Pp  are  sorted  non-decreasingly,  then  the  product  matrix  {oiipj) 
is  an  Anti-Monge  matrix. 

In  contrast  with  the  traveling  salesman  problem,  it  turns  out  that  the  QAP 
with  both  coefficient  matrices  being  Monge  or  Anti-Monge  is  NP-hard,  whereas 
the  complexity  of  a QAP  with  one  coefficient  matrix  being  Monge  and  the  other 
one  being  Anti-Monge  is  still  open,  see  Burkard,  Qela,  Demidenko,  Metelski,  and 
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Woeginger  [29]  and  Qela  [45],  However,  some  polynomially  solvable  special  cases 
can  be  obtained  by  imposing  additional  conditions  on  the  coefficient  matrices. 
These  special  cases  involve  very  simple  matrices  like  product  matrices  or  so-called 
chess-board  matrices.  A matrix  A = (aij)  is  a chess-board  matrix  if  its  entries  are 
given  by  = (— 1)*+A  These  QAPs  can  either  be  formulated  as  equivalent  LAPs, 

or  they  are  constant  permutation  QAPs  (see  [29,  45]),  that  is,  their  optimal  solution 
can  be  given  before  hand,  without  knowing  the  entries  of  their  coefficient  matrices. 
A few  other  versions  of  the  QAP  involving  Monge  and  Anti-Monge  matrices  with 
additional  structural  properties  can  be  solved  by  dynamic  programming. 

Other  restricted  versions  of  the  QAP  involve  matrices  with  a specific  diagonal 
structure  such  as  circulant  and  Toeplitz  matrices.  An  n x n matrix  A = (aij) 
is  called  a Toeplitz  matrix  if  there  exist  numbers  c_„+i,  . . . , c_i,  cq,  Ci,  . . . , c„_i 
such  that  Oij  = Cj_j,  for  all  i,j. 

A matrix  A is  called  a circulant  matrix  if  it  is  a Toeplitz  matrix  and  the 
generating  numbers  c*  fulfill  the  conditions  c,-  = c„_j,  for  0 < i < n - 1.  In  other 
words,  a Toeplitz  matrix  has  constant  entries  along  lines  parallel  to  the  diagonal, 
whereas  a circulant  is  given  by  its  first  row  and  the  entries  of  the  i— th  row  resembles 
the  first  row  shifted  by  z — 1 places  to  the  right. 

QAPs  with  one  Anti-Monge  (Monge)  matrix  and  one  Toeplitz  (circulant)  ma- 
trix. In  general  these  versions  of  the  QAP  remain  NP-hard  unless  additional  condi- 
tions, such  as  monotonicity,  are  imposed  on  the  coefficient  matrices.  A well  studied 
problem  is  the  so  called  Anti-Monge-Toeplitz  QAP  where  the  rows  and  columns 
of  the  Anti-Monge  matrix  are  non-decreasing,  investigated  by  Burkard,  Qela,  Rote 
and  Woeginger  [30].  It  has  been  shown  that  this  problem  is  NP-hard  and  contains 
as  a special  case  the  so  called  turbine  problem  introduced  by  Mosewich  [141]  and 
formulated  as  a QAP  by  Laporte  and  Mercure  [121].  In  the  turbine  problem  we 
are  given  a number  of  blades  to  be  welded  in  regular  spacing  around  the  cylinder 
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of  the  turbine.  Due  to  inaccuracies  in  the  manufacturing  process  the  weights  of 
the  blades  differ  slightly  and  consequently  the  gravity  center  of  the  system  does 
not  lie  on  the  rotation  axis  of  the  cylinder,  leading  to  instabilities.  In  an  effort 
to  make  the  system  as  stable  as  possible,  it  is  desirable  to  locate  the  blades  so 
as  to  minimize  the  distance  between  the  center  of  gravity  and  the  rotation  axis. 
The  mathematical  formulation  of  this  problem  leads  to  an  NP-hard  Anti-Monge- 
Toeplitz  QAP.  (For  more  details  and  for  a proof  of  NP-hardness  see  Burkard  et 
al.  [30].)  It  is  probably  interesting  that  the  maximization  version  of  this  prob- 
lem is  polynomially  solvable.  Further  polynomially  solvable  special  cases  of  the 
Anti-Monge-Toeplitz  QAP  arise  if  additional  constraints  such  as  benevolence  or 
k-benevolence  are  imposed  on  the  Toeplitz  matrix.  These  conditions  are  expressed 
in  terms  of  properties  of  the  generating  function  of  these  matrices,  see  Burkard  et 
al.  [30]. 

The  polynomially  solvable  QAPs  with  one  Anti-Monge  (Monge)  matrix  and 
the  other  one  Toeplitz  (circulant)  matrix  described  above,  are  all  constant  permu- 
tation QAPs.  The  techniques  used  to  prove  this  fact  and  to  identify  the  optimal 
permutation  is  called  reduction  to  extremal  rays.  This  technique  exploits  two  facts: 
first,  the  involved  matrix  classes  form  cones,  and  secondly,  the  objective  function  of 
the  QAP  is  linear  with  respect  to  each  of  the  coefficient  matrices.  These  two  facts 
allow  us  to  restrict  the  investigations  to  instances  of  the  QAP  with  0-1  coefficient 
matrices  which  are  extremal  rays  of  the  above  mentioned  cones.  Such  instances 
can  then  be  handled  by  elementary  means  (exchange  arguments,  bounding  tech- 
niques) more  easily  that  the  general  given  QAP.  The  identification  of  polynomially 
solvable  special  cases  of  the  QAP  which  are  not  constant  permutation  QAPs  and 
can  be  solved  algorithmically  remains  a challenging  open  question. 

Another  class  of  matrices  similar  to  the  Monge  matrices  are  the  Kalmanson 
matrices.  A matrix  A = (a^)  is  a Kalmanson  matrix  if  it  is  symmetric  and  its 
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elements  satisfy  the  following  inequalities  for  all  indices  i,j,  k,l,  i < j < k < 1: 


Q>ij  ^kl  ^ ^ik  ^jk  ^ ^ik  ^jl' 

For  more  information  on  Monge,  Anti-Monge  and  Kalmanson  matrices,  and 
their  properties  the  reader  is  referred  to  the  survey  article  of  Burkard,  Klinz  and 
Rudolf  [37].  The  Koopmans-Beckmann  QAP  with  one  coefficient  matrix  being  is  a 
Kalmanson  matrix  and  the  other  one  a Toeplitz  matrix,  has  been  investigated  by 
Demeko  and  Woeginger  [57].  The  computational  complexity  of  this  problem  is  an 
open  question,  but  analogously  as  in  the  case  of  the  Anti-Monge-Toeplitz  QAP, 
polynomially  solvable  versions  of  the  problem  are  obtained  by  imposing  additional 
constraints  to  the  Toeplitz  matrix. 

Further  polynomially  solvable  cases  arise  as  QAP  formulations  of  other  prob- 
lems, like  the  linear  arrangement  problem,  minimum  feedback  arc  set  problem, 
packing  problems  in  graphs  and  subgraph  isomorphism,  see  [29,  45].  Polynomially 
solvable  versions  of  these  problems  lead  to  polynomially  solvable  cases  of  the  QAP. 
The  coefficient  matrices  of  these  QAPs  are  the  (weighted)  adjacency  matrices  of 
the  underlying  graphs,  and  the  special  structure  of  these  matrices  is  imposed  by 
properties  of  these  graphs.  The  methods  used  to  solve  these  QAPs  range  from 
graph  theoretical  algorithms  (in  the  case  of  the  linear  arrangement  problem  and 
the  feedback  arc  set  problem),  to  dynamic  programming  (in  the  case  of  subgraph 
isomorphism). 


2.12  QAP  Problem  Generators 

Since  the  QAP  is  a very  hard  problem  from  a practical  point  of  view,  often 
heuristics  are  the  only  reasonable  approach  to  solve  it,  and  so  far  there  exists  no 
performance  guarantees  for  any  of  the  algorithms  developed  for  the  QAP.  One  pos- 
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sibility  to  evaluate  the  performance  of  heuristics  and  to  compare  different  heuristics 
is  given  by  QAP  instances  with  known  optimal  solution.  Heuristics  are  applied  to 
these  instances  and  the  heuristic  solution  is  compared  to  the  optimal  one  known 
before  hand.  The  instances  with  known  optimal  solution  should  ideally  have  two 
properties;  first,  they  should  be  representative  in  terms  of  their  hardness,  and 
secondly,  they  should  not  be  especially  easy  for  any  of  the  heuristics. 

Two  generators  of  QAP  instances  with  known  optimal  solution  have  been  pro- 
posed so  far:  Palubeckis’  generator  [151]  and  the  generator  proposed  by  Li  and 
Pardalos  [127]. 

The  first  method  for  generating  QAP  instances  with  a known  optimal  solution 
was  proposed  by  Palubeckis  [151]  in  1988.  The  input  of  the  Palubeckis’  algorithm 
consists  of  the  size  n of  the  instance  to  be  generated,  the  optimal  solution  (permu- 
tation) p of  the  output  instance,  two  control  parameters  w and  z,  where  z < w, 
and  the  distance  matrix  A of  an  r x s grid  with  rs  = n.  A contains  rectilinear 
distances  also  called  Manhattan  distances,  that  is,  the  distance  between  two 
given  knots  i,  j lying  in  rows  rj,  rj  and  in  columns  Cj,  cj,  respectively,  is  given  by 


aij  — I r j T'j  I -f-  I C{ 


. The  output  of  the  algorithm  is  a second  matrix  B such 


that  p is  an  optimal  solution  of  QAP{A,  B).  The  idea  is  to  start  with  a matrix 
B such  that  QAP{A,  B)  is  a trivial  instance  with  optimal  solution  p.  Then  B is 
transformed  such  that  QAP{A,  B)  is  not  any  more  trivial,  but  p continues  to  be 
its  optimal  solution. 

Palubeckis  starts  with  a constant  matrix  B = {hij)  with  bij  = w.  QAP{A,  B) 
is  a trivial  problem  because  all  permutations  yield  the  same  value  of  the  objective 
function  and  thus,  are  optimal  solutions.  Hence,  also  the  identity  permutation  id 
is  an  optimal  solution  of  QAP  (A,  B).  Then  matrix  B is  iteratively  transformed  so 
that  it  is  not  a constant  matrix  any  more  and  the  identity  permutation  remains 
an  optimal  solution  of  QAP{A,B).  In  the  last  iteration  the  algorithm  constructs 
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an  instance  QAP{A',  B)  with  optimal  solution  p with  the  help  of  QAP{A,  B)  with 
optimal  solution  id,  by  setting  A'  - {ap(i)p{j)).  The  optimal  value  of  QAP{A',B) 
equals  w ^ dij- 

Cyganski,  Vaz  and  Virball  [55]  have  observed  that  the  QAP  instances  generated 
by  Palubeckis’  generator  are  “easy”  in  the  sense  that  their  optimal  value  can  be 
computed  in  polynomial  time  by  solving  a linear  program.  (For  an  accessible  proof 
of  this  result  the  reader  is  referred  to  [45].)  Notice,  however,  that  nothing  is  known 
about  the  computational  complexity  of  QAP  instances  generated  by  Palubeckis’ 
generator.  We  believe  that  finding  an  optimal  solution  for  these  QAPs  is  NP-hard, 
although  the  corresponding  decision  problem  is  polynomially  solvable. 

Another  generator  of  QAP  instances  with  known  solution  has  been  proposed 
by  Li  and  Pardalos  [127].  As  Palubeckis’  generator,  Li  and  Pardalos  starts  with  a 
trivial  instance  QAP{A,  B)  with  the  identity  permutation  id  as  optimal  solution 
and  iteratively  transforms  A and  B so  that  the  resulting  QAP  instance  still  has 
the  optimal  solution  id  but  is  not  trivial  any  more.  The  transformations  are  such 
that  for  all  i,j,  i',f,  Oij  > ai'j'  is  equivalent  to  bij  < at  the  end  of  each  iteration. 

If  the  coefficient  matrices  are  considered  as  weighted  adjacency  matrices  of 
graphs,  each  iteration  transforms  entries  corresponding  to  some  specific  subgraph 
equipped  with  signs  on  the  edges  and  hence  called  sign-subgraphs.  The  application 
of  Li  and  Pardalos’  algorithm  with  different  sign-subgraphs  yields  different  QAP 
generators.  A number  of  generators  involving  different  sign-subgraphs,  for  ex- 
ample, subgraphs  consisting  of  a single  edge,  signed  triangles  and  signed  spanning 
trees  have  been  tested.  It  is  perhaps  interesting  and  surprising  that  QAP  instances 
generated  by  involving  more  complex  sign-subgraphs  are  generally  “easier”  than 
those  generated  by  involving  subgraphs  consisting  of  single  edges.  Here  a QAP 
instance  is  considered  to  be  “easy” , if  most  heuristics  applied  to  it  find  a solution 
near  to  the  optimal  one  in  a relatively  short  time.  Nothing  is  known  about  the 
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complexity  of  QAP  instances  generated  by  the  generator  of  Li  and  Pardalos,  since 
the  arguments  used  to  analyze  Palubeckis’  generator  do  not  apply  in  this  case. 

2.13  Available  Computer  Codes  for  the  QAP 

Burkard,  Karisch,  and  Rendl  [36]  have  compiled  a library  of  QAP  instances 
(QAPLIB)  which  is  widely  used  to  test  bounds,  exact  algorithms,  and  heuristics 
for  the  QAP.  The  instances  collected  in  QAPLIB  are  due  to  different  authors 
and  range  from  instances  arising  in  real  life  applications  to  instances  generated 
randomly  only  for  test  purposes.  Many  of  these  instances  have  not  been  solved 
to  optimality  yet,  the  most  celebrated  among  them  being  the  instances  of  Nu- 
gent, Vollmann,  and  Ruml  [147]  of  size  larger  than  25.  QAPLIB  can  be  found  at 
http ; //www . opt . math . tu-graz . ac . at/~kar isch/qaplib. 

A number  of  codes  to  compute  lower  bounds  are  available.  A FORTRAN  code 
which  computes  the  GLB  is  due  to  Burkard  and  Derigs  [31],  and  is  able  to  compute 
the  bound  for  instances  of  size  up  to  256.  The  source  code  can  be  downloaded  from 
the  QAPLIB  web  page.  Another  FORTRAN  code  which  can  be  downloaded  from 
the  QAPLIB  web  page  computes  the  elimination  bound  (ELI)  for  symmetric  QAP 
instances  of  size  up  to  256. 

Recently,  Espersen,  Karisch,  Qela,  and  Clausen  [66]  have  developed  QAPpack 
which  is  a JAVA  package  containing  a branch  and  bound  algorithm  to  solve  the 
QAP.  In  QAPpack  a number  of  bounds  based  on  linearization  are  implemented: 
the  Gilmore-Lawler  bound  [81,  123],  the  bound  of  Carraresi  and  Malucelli  [44], 
the  bound  of  Adams  and  Johnson  [3],  the  bound  of  Hahn  and  Grant  [94],  and  the 
bound  of  Karisch,  Qela,  Clausen,  and  Espersen  [108].  The  implementation  is  based 
on  the  dual  framework  provided  by  Karisch  et  al.  [108].  QAPpack  can  be  found  at 
http : / / WWW . imm . dtu . dk/~te/QAPpack. 
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Besides  QAPpack,  a FORTRAN  code  of  the  branch  and  bound  algorithm  de- 
veloped by  Burkard  and  Derigs  [31]  can  be  downloaded  from  the  QAPLIB  web 
page. 

There  are  also  some  codes  of  heuristics  available.  The  source  files  (compressed 
tar-files)  of  two  FORTRAN  implementations  of  GRASP  for  dense  QAPs  by  Re- 
sende,  Pardalos  and  Li  [129]  and  sparse  QAPs  by  Pardalos,  Pitsoulis  and  Re- 
sende  [154]  can  be  downloaded  from  Resende’s  web  page  at 
http : //www . research . att . com/~mgcr/src/index . html. 

The  source  file  of  a FORTRAN  implementation  of  the  simulated  annealing 
algorithm  of  Burkard  and  Rendl  [38]  can  be  downloaded  from  the  QAPLIB  web 
page. 

The  source  file  of  a C-|— I-  implementation  of  the  simulated  annealing  algorithm 
of  Connolly  [52],  due  to  Taillard,  can  be  downloaded  from  Taillard’s  web  page  at 
http : / / WWW . idsia . ch/~eric/codes . dir/sa_qap . c. 

Also  a source  file  of  a PASCAL  implementation  of  Taillard’s  robust  tabu  search 
[182]  can  be  found  at  Taillard’s  web  page. 

Finally,  the  source  file  of  a FORTRAN  implementation  of  Li  and  Pardalos’ 
generator  for  QAP  instances  with  known  optimal  solution  [127]  can  be  obtained 
by  sending  an  email  to  coapmath.ufl.edu  with  subject  line  send  92006. 


CHAPTER  3 

SPARSE  QUADRATIC  ASSIGNMENT  PROBLEMS 


3.1  Introduction 

In  this  chapter  we  present  a version  of  the  GRASP  for  solving  problem  instances 
of  QAP{F,  D)  as  defined  in  §2.1,  where  one  of  the  input  matrices  is  sparse.  We 
define  the  sparsity  of  a given  matrix  M to  be: 

number  of  zeros  in  M 
^ number  of  entries  in  M 

We  denote  the  sparsity  of  an  instance  QAP{F,D)  by  ma,x{S p , S d) ■ We  use  this 
definition  because  the  algorithm  benefits  if  at  least  one  of  the  input  matrices  F 
or  D is  sparse.  Sparse  quadratic  assignment  problems  arise  in  many  practical 
applications,  including  data  analysis  [99]  and  multiprocessor  scheduling  [22].  In 
§3.2,  the  sparse  code  (GRASP-S)  is  described.  The  design  and  implementation 
of  gqaps.f,  a set  of  FORTRAN  subroutines  for  the  solving  sparse  instances  of 
the  QAP,  is  presented  in  §3.3.  Usage  of  the  code  is  described  in  §3.4.  In  §3.5, 
we  compare  GRASP-S  with  the  dense  implementation  of  GRASP  for  the  QAP 
(GRASP-D)  presented  in  Li  et  al.  [130,  129],  on  a large  set  of  test  problems  from 
the  QAPLIB  [36]. 

3.2  GRASP  for  Sparse  Quadratic  Assignment  Problems 

The  construction  phase  for  the  GRASP-S  exploits  the  sparsity  of  either  one  of 
the  matrices  F and  D,  to  reduce  the  computational  effort  required  to  construct 
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an  approximate  solution.  The  construction  phase  is  composed  of  a sparse  and  a 
dense  procedures. 

To  describe  the  sparse  procedure,  presented  in  pseudo-code  form  in  Figure  3.1, 
we  need  the  following  definitions: 

• r is  the  set  of  already-made  assignments, 

• A is  the  set  of  zero-cost  assignment  pairs  with  respect  to  the  first  two  as- 
signments. 


A;={(i,A:)|  ^4/ = 0,  |r|  = 2}. 

U,i)er 

A'  is  the  set  of  zero-cost  assignment  pairs  with  respect  to  the  last  element  in 
the  set  r. 


A'  :=  {(z,  = 0,  |r|  > 3,  (i|r|,/|r|)  € T}. 


The  two  assignments  made  in  the  initial  construction  phase  are  input  to  the 
sparse  GRASP  construction  phase.  Since  the  first  two  assignments  are  randomly 
chosen  among  the  smallest  cost  assignments,  if  their  resulting  cost  is  not  zero, 
the  problem  data  has  little  or  no  sparsity,  and  we  disregard  the  sparse  procedure. 
Otherwise,  in  lines  2-3  the  appropriate  initializations  for  sets  F and  A are  made. 
In  lines  4-12  the  set  A is  constructed,  consisting  of  all  zero-cost  assignments  with 
respect  to  the  initial  two  assignments.  From  A,  a random  selection  of  the  third 
assignment  pair  to  be  added  to  the  set  F is  made  in  lines  13-16. 


procedure  ConstructSparse((A, ^i),  {32ih)) 

1 

if  # 0 ^ return; 

2 

r = {(ii,  ^i),  O2,  ^2)}; 

3 

A = 0; 

4 

do  i = 1,  . . . j 77/  — V 

5 

do  J = 1,  2j  . . . ,77/  — ^ 

6 

if  (i,  k)^T-^ 

7 

if  fij^ki  = 0 

8 

A = AU{(Z,A:)}; 

9 

fi; 

10 

fi; 

11 

od; 

12 

od; 

13 

s = |A|; 

14 

r = random[l,  s]; 

15 

(i,  k)  = r— th  element  of  set  A; 

16 

r = ru{(i,A:)}; 

17 

do  assignments=  4, . . . , n ^ 

18 

A'  = 0; 

19 

do  2 = 1,2,...  ,77/  — y 

20 

do  k — 1,2,...  ^ Ti  — y 

21 

if  (i,  A:)  e A \ r -> 

22 

if  /ij|n^W|r|  = 0 

23 

A'  = A'U{(i,A:)}; 

24 

fi; 

25 

fi; 

26 

od; 

27 

od; 

28 

if  A'  = 0 — > return; 

29 

A = A'; 

30 

s = |A|; 

31 

r = random[l,  s]; 

32 

{i,  k)  — r— th  element  of  set  A; 

33 

r = ru  {(z,  A:)}; 

34 

od; 

end  ConstructSparse; 

Figure  3.1:  GRASP-S  construction  phase  pseudo-code 
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The  effect  of  sparsity  on  the  computational  effort  is  illustrated  in  lines  18-27, 
where  it  is  unnecessary  to  check  all  possible  assignments  that  result  in  zero-cost 
with  respect  to  the  assignments  in  set  P,  but  only  those  associated  with  the  last 
inserted  element  of  set  P.  Another  set  of  zero-cost  assignments  A'  is  constructed, 
and  the  procedure  is  repeated  until  no  further  zero-cost  assignments  can  be  made 
(A'  = 0).  The  size  of  the  set  of  zero-cost  assignments  is  decreased  each  time  an 
assignment  is  made.  In  lines  30-33,  the  algorithm  randomly  chooses  an  assignment 
pair  to  be  added  to  P. 

It  is  clear  that  the  performance  of  the  above  algorithm  improves  as  the  sizes 
of  A and  A'  increase.  The  number  of  zero-cost  assignment  pairs  depends  both  on 
the  sparsity  and  the  structure  of  the  problem. 

The  sparse  construction  procedure  terminates  when  the  set  A'  = 0,  that  is, 
there  are  no  zero-cost  assignment  pairs  left.  Upon  termination,  the  set  P will 
contain  m zero-cost  assignment  pairs,  and  the  remaining  n — m assignments  must 
be  made.  This  is  done  by  passing  the  partial  set  P to  the  dense  construction 
procedure  presented  in  §2.10.7.  The  construction  procedure  starts  with  the  m 
already-made  assignments  of  the  sparse  procedure,  and  proceeds  with  the  rest  of 
the  assignments  until  |P|  = n.  For  each  assignment,  it  produces  all  the  possible 
pairs  and  keeps  only  the  ones  that  have  not  been  assigned.  For  those  assignment 
pairs,  it  inserts  their  corresponding  cost  Cms  into  a heap,  and  updates  a counter  U, 
which  identifies  the  number  of  elements  in  the  heap.  This  process  is  illustrated  in 
lines  4-12  of  the  pseudo-code  in  Figure  4.3.  In  lines  13-16,  the  algorithm  chooses 
randomly  an  assignment  pair  from  the  all  pairs,  which  make  up  the  restricted 
candidate  list.  In  line  17,  the  chosen  assignment  pair  is  added  into  the  set  P and 
the  algorithm  proceeds  with  the  next  iteration.  After  the  dense  procedure  has 
finished,  a constructed  solution  is  on  hand,  and  local  search  is  applied  to  improve 
the  existing  solution.  The  local  search  used  is  described  in  Li  et  al.  [129]. 
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3.3  Design  and  Implementation  of  the  Subroutines 

The  code  is  implemented  in  ANSI  standard  FORTRAN  77,  and  runs  without 
any  modifications  on  UNIX  environments.  It  should  also  run  on  other  environ- 
ments with  little  or  no  change.  Input,  output,  variable  and  array  declarations  and 
parameter  settings  are  all  performed  in  a separate  driver  module.  Therefore  usage 
and  experimentation  with  the  code  is  relatively  simple. 

Subroutine  gqaps  is  called  from  the  driver  module  which  in  addition  to  a num- 
ber of  auxiliary  arrays  passes  the  following  input; 

n:  Size  of  QAP. 

f,  d:  QAP  fiow  and  distance  matrices. 
a,  (5:  Restricted  candidate  list  parameters, 
maxiter : Maximum  number  of  GRASP  iterations, 
seed:  Random  number  generator  seed. 

look4:  A target  value  beyond  which  further  improvement  may  not  be  necessary. 
A value  of  -1  will  force  the  optimizer  to  stop  at  the  maximum  number  of  iterations. 

Subroutine  gqaps  performs  the  optimization,  and  returns  the  best  permuta- 
tion (opta)  with  the  corresponding  minimum  cost  (bestv)  and  the  number  of 
iterations  performed  (iter).  Within  the  GRASP  iterations  three  main  functions 
are  performed.  Namely,  the  stage  1 (buildO)  creates  the  first  two  initial  assign- 
ments simultaneously,  stage  2 (buildl)  constructs  a complete  solution  by  assigning 
the  rest  n — 2 assignments,  and  a 2-exchange  local  search  (local)  is  done  in  the 
constructed  solution  for  further  improvement.  Before  GRASP  proceeds  to  the 
next  iteration,  it  checks  the  cost  of  the  current  solution  (objv)  and  keeps  the  best 
(bestv)  among  all  iterations  (savsol). 
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Since  stage  1 of  the  construction  phase  uses  the  same  sorted  assignment  costs 
during  each  GRASP  iteration,  the  costs  are  sorted  once,  outside  the  main  GRASP 
loop,  in  subroutine  srtcst.  The  subroutine  uses  heap  sort  to  partially  sort  the  flow 
values,  distance  values,  and  assignment  costs.  The  first  (double)  loops  insert  off- 
diagonal  distance  and  flow  values  into  their  respective  priority  heaps.  Parameter 
nn  is  computed  and,  in  the  next  do  loop,  the  smallest  distance  value  and  largest 
flow  values  are  removed  from  the  heaps,  and  the  assignment  cost  is  computed  and 
inserted  into  its  priority  heap.  That  do  loop  is  repeated  times.  The  final  do  loop 
puts  into  array  cost  the  smallest  assignment  costs  by  sequentially  removing 
from  the  assignment  cost  priority  heap  the  smallest  cost  element.  Information 
needed  to  retrieve  the  assignments  corresponding  to  each  cost  is  put  in  the  array 
fdind. 

Subroutine  buildO  implements  stage  1 of  the  GRASP  construction  phase.  The 
permutation  arrays  a and  b are  initialized,  the  index  (nselct)  of  the  assignment 
pair  is  chosen  at  random,  and  the  assignment  indices  i,  j,  k,  1 are  recovered  from 
the  data  structure.  Just  prior  to  the  second  do  loop,  the  first  assignment  is  put 
in  the  permutation  arrays  and,  in  the  last  two  loops,  the  second  assignment  is 
put  in  the  arrays.  Since  the  first  two  assignments  are  chosen  randomly  among  the 
smallest,  if  the  resulting  cost  in  not  equal  to  zero,  then  the  problem  contains  little 
or  no  sparsity. 

Subroutine  buildl  implements  stage  2 of  the  GRASP  construction  phase.  This 
subroutine  is  divided  into  two  phases.  The  first  phase  is  performed  if  the  cost 
resulting  from  the  initial  two  assignments  is  zero,  where  it  proceeds  by  constructing 
an  array  cp  that  contains  all  the  assignment  pairs  (facility  i to  location  j)  which 
result  in  zero  cost  with  respect  to  the  already  made  assignments.  Every  time  a 
new  assignment  pair  is  added  to  the  set  of  already  made  assignments,  array  cp  is 
updated  to  reflect  the  new  assignment,  and  the  number  of  zero  assignment  pairs 
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assz  is  incremented  by  one.  Phase  1 will  terminate  when  there  is  no  assignment 
pair  that  will  result  in  zero  cost,  in  which  case  buildl  proceeds  to  the  second 
phase. 

In  phase  2,  the  first  outer  do  loop  makes  assignments  assz+1, . . . ,n  — 1.  In 
that  loop,  the  cost,  with  respect  to  previously-assigned  pairs,  of  each  possible 
assignment  is  computed.  The  cost  is  inserted  in  its  priority  heap  for  sorting. 
After  all  costs  have  been  computed,  the  index  (nselct)  of  the  randomly  selected 
assignment  is  determined  and  the  assignment  is  retrieved  from  the  heap.  In  both 
phases  of  buildl,  subroutine  mkassg  updates  permutation  arrays  a and  b with  the 
latest  assignment.  The  cost  of  the  last  assignment  is  added  to  the  total  assignment 
cost  in  the  final  do  loop. 

Subroutine  local  implements  the  2-exchange  of  the  GRASP  local  search  phase. 
The  local  search  is  carried  out  on  permutation  array  a,  so  a rearrangement  is  done 
in  subroutine  mkbseq  to  make  permutation  array  b = {1, 2, . . . , n}.  In  the  (double) 
loops,  for  all  pairs  i,  j such  that  j > i,  subroutine  evalij  evaluates  the  gain 
xgain  of  swapping  i and  j in  permutation  array  a.  If  a positive  gain  is  attained, 
the  swap  is  carried  out  and  the  cost  of  the  assignment  (objv)  is  updated.  The 
procedure  ends  when  no  further  improvement  is  possible  by  swapping  elements  in 
the  permutation  array. 

Subroutine  savsol  saves  permutation  a in  array  opt  a and  the  cost  of  the 
assignment  (objv)  in  scalar  bestv. 

In  subroutine  insrtq,  a pair  of  elements  (v  and  iv)  is  inserted  into  a priority 
heap  of  pairs  (q  and  iq),  ordered  by  the  values  in  q.  The  size  of  the  heap  (sizeq) 
is  updated.  The  heap  is  updated  to  take  into  account  the  additional  elements. 

In  subroutine  removq,  a pair  of  elements  (v  and  iv)  is  removed  from  a priority 
heap  of  pairs  (q  and  iq),  ordered  by  the  values  in  q.  The  size  of  the  heap  (sizeq) 
is  updated.  The  heap  is  updated  to  take  into  account  the  removal  of  the  elements. 
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Subroutine  randp  is  the  portable  random  number  generator  of  Schrage  [176]. 
Given  a seed  ix  G [0,2^^  — 1],  it  returns  by  parameter  a new  seed  in  the  same 
range  and  by  value  a real  number  in  the  interval  [0,1]. 

Subroutine  evalij  evaluates  the  assignment  cost  gain  attained  by  swapping 
indices  i and  j in  an  assignment  permutation  array  a.  Index  j is  assumed  to  be 
greater  than  i.  The  do  loop  computes  the  interaction  with  the  other  facilities. 

Subroutine  mkbseq  applies  identical  transformations  to  arrays  a and  b to  make 
b = {1, 2, . . . , n}. 


3.4  Usage  of  the  Subroutines 

The  subroutines  in  file  gqaps . f carry  out  the  approximate  optimization  of  the 
QAP.  The  user  interface  with  them  is  subroutine  gqaps,  that  must  be  called  from 
a driver  program. 

The  sample  driver  program  for  gqaps  included  in  the  distribution  (driver. f), 
is  set  for  problems  of  dimension  n < 256.  All  variables  and  arrays  needed  by 
subroutine  gqaps  are  defined.  Variable  iseedO,  used  by  the  driver,  is  also  defined. 
Subroutines  readp  and  outsol  are  examples  of  code  that  can  be  used  for  input 
and  output,  respectively. 

As  an  example,  consider  the  QAPLIB  instance  escl6e  with  input  file  shown 
in  Figure  3.2.  Running  the  driver  program  for  instance  esc64a  of  dimension  64 
produces  the  output  shown  in  Figure  3.3. 

3.5  Computational  Results 

The  GRASP-S  FORTRAN  implementation  was  tested  on  a Silicon  Graphics 
Challenge  computer,  using  the  f 77  compiler  with  flags  -02  -Olimit  800.  The  set 
of  problems  consisted  of  all  problems  in  QAPLIB  [36]  of  dimension  n > 8 that  are 
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16 

0011011110000000 

0001000000000000 

1000000000000000 

1100011110000000 

0000011110000000 

1001102320000000 

1001120210000000 

1001132020000000 

1001121200000000 

0000000000000000 

0000000000000000 

0000000000000000 

0000000000000000 

0000000000000000 

0000000000000000 

0000000000000000 

0001011201121223 

0010102110212132 

0100120112012312 

1000211021103221 

0112000112230112 

1021001021321021 

1201010023121201 

2110100032212110 

0112122300010112 

1021213200101021 

1201231201001201 

2110322110002110 

1223011201120001 

2132102110210010 

2312120112010100 

3221211021101000 


Figure  3.2:  Input  file  of  QAPLIB  instance  escl6e. 


116 


GRASP  for  QAP 


input 

dimension  of  qap  : 64 
construction  phase  parameter  alpha  : 0.25 
construction  phase  parameter  beta  : 0.50 
sparsity  of  matrix  f : 0.09 
sparsity  of  matrix  d : 0.95 
maximum  number  of  grasp  iterations  : 100 
look4  : 116 
initial  seed  : 270001 


output 

grasp  iterations 

cost  of  best  permutation  found 

best  permutation  found 


1 

116 


16 

24 

1 

46 

31 

58 

52 

60 

21 

42 

10 

33 

35 

30 

62 

49 

18 

13 

50 

45 

6 

22 

15 

9 

19 

14 

41 

4 

53 

2 

61 

20 

8 

17 

7 

32 

26 

11 

43 

56 

55 

36 

40 

25 

12 

28 

48 

64 

39 

23 

34 

63 

38 

51 

37 

59 

5 

47 

29 

44 

3 

27 

54 

57 

Figure  3.3:  Sample  output  of  driver  for  QAP  instance  esc64a 
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Table  3.1:  Summary  of  runs  for  32  and  2048  iterations  ( n < 16) 

32  iterations  2048  iterations 


name 

bks 

Sm 

min 

avg 

max 

min 

avg 

max 

chrl2a 

9552 

0.76 

9552 

10088 

10688 

9552 

9552 

9552 

chrl2b 

9742 

0.76 

9742 

9742 

9742 

9742 

9742 

9742 

chrl2c 

11156 

0.76 

11758 

12181 

12858 

11156 

11156 

11156 

chrl5a 

9896 

0.8 

10624 

11118 

11606 

9896 

9920 

9978 

chrl5b 

7990 

0.8 

7990 

8920 

9552 

7990 

7990 

7990 

chrl5c 

9504 

0.8 

10748 

11892 

12704 

9504 

9788 

9940 

escOSa 

2 

0.78 

2 

2 

2 

2 

2 

2 

escOSb 

8 

0.56 

8 

8 

8 

8 

8 

8 

escOSc 

32 

0.437 

32 

32 

32 

32 

32 

32 

escOSd 

6 

0.56 

6 

6 

6 

6 

6 

6 

escOSe 

2 

0.687 

2 

2 

2 

2 

2 

2 

escOSf 

18 

0.375 

18 

18 

18 

18 

18 

18 

esc 16a 

68 

0.64 

68 

68 

68 

68 

68 

68 

escl6b 

292 

0.25 

292 

292 

292 

292 

292 

292 

esc 16c 

160 

0.53 

160 

160 

160 

160 

160 

160 

escl6d 

16 

0.77 

16 

16 

16 

16 

16 

16 

escl6e 

28 

0.77 

28 

28 

28 

28 

28 

28 

escl6f 

0 

0.93 

0 

0 

0 

0 

0 

0 

escl6g 

26 

0.77 

26 

26 

26 

26 

26 

26 

escl6h 

996 

0.25 

996 

996 

996 

996 

996 

996 

escl6i 

14 

0.82 

14 

14 

14 

14 

14 

14 

escl6j 

8 

0.84 

8 

8 

8 

8 

8 

8 

lipalOa 

473 

0 

473 

473 

473 

473 

473 

473 

lipalOb 

2008 

0 

2008 

2008 

2008 

2008 

2008 

2008 

nug08 

214 

0.31 

214 

214 

214 

214 

214 

214 

nugl2 

578 

0.37 

578 

584 

594 

578 

578 

578 

nugl5 

1150 

0.33 

1150 

1154 

1166 

1150 

1150 

1150 

roulO 

174220 

0 

174220 

174220 

174220 

174220 

174220 

174220 

roul2 

235528 

0 

235528 

237935 

240124 

235528 

235528 

235528 

roul5 

354210 

0 

354210 

361850 

366930 

354210 

354210 

354210 

scrlO 

26992 

0.56 

26992 

26992 

26992 

26992 

26992 

26992 

scrl2 

31410 

0.61 

31410 

31410 

31410 

31410 

31410 

31410 

GRASP-D  time 
GRASP-S  time 


0 0.2  0.4  0.6  0.8  1 

sparsity 


Figure  3.4:  Solution  time  ratios  as  a function  of  problem  sparsity  (32  iterations). 
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Table  3.2:  Summary  of  runs  for  32  and  2048  iterations  ( 18  < n < 36) 

32  iterations  2048  iterations 


name 

bks 

Sm 

min 

avg 

max 

min 

avg 

max 

chrlSa 

11098 

0.83 

13258 

14014 

15290 

11098 

11392 

11996 

chrlSb 

1534 

0.83 

1556 

1612 

1644 

1534 

1534 

1534 

chr20a 

2192 

0.85 

2518 

2626 

2704 

2232 

2331 

2422 

chr20b 

2298 

0.85 

2664 

2756 

2906 

2374 

2452 

2514 

chr20c 

14142 

0.85 

14996 

17460 

18594 

14142 

14142 

14142 

chr22a 

6156 

0.87 

6528 

6600 

6736 

6230 

6302 

6378 

chr22b 

6194 

0.87 

6448 

6610 

6788 

6380 

6408 

6424 

chr25a 

3796 

0.88 

4810 

5009 

5240 

3954 

4282 

4386 

elsl9 

17212548 

0.68 

17212548 

17792100 

17937024 

17212548 

17212548 

17212548 

esc32a 

130 

0.82 

142 

145 

148 

134 

135 

138 

esc32b 

168 

0.75 

184 

190 

196 

168 

168 

168 

esc32c 

642 

0.8 

642 

642 

642 

642 

642 

642 

esc32d 

200 

0.85 

200 

200 

202 

200 

200 

200 

esc32e 

2 

0.85 

2 

2 

2 

2 

2 

2 

esc32f 

2 

0.87 

2 

2 

2 

2 

2 

2 

esc32g 

6 

0.91 

6 

6 

6 

6 

6 

6 

esc32h 

438 

0.9 

440 

440 

440 

438 

438 

438 

kra30a 

88900 

0.63 

90860 

92126 

93730 

88900 

89538 

90090 

kra30b 

91420 

0.6 

92650 

93246 

94090 

91420 

91544 

91650 

lipa20a 

3683 

0 

3683 

3720 

3766 

3683 

3683 

3683 

lipa20b 

27076 

0 

27076 

27076 

27076 

27076 

27076 

27076 

lipa30a 

13178 

0 

13389 

13399 

13414 

13178 

13241 

13343 

lipa30b 

151426 

0 

151426 

156148 

175037 

151426 

151426 

151426 

nug20 

2570 

0.3 

2612 

2620 

2632 

2570 

2570 

2574 

nug30 

6124 

0.2 

6196 

6222 

6244 

6136 

6143 

6152 

rou20 

725522 

0 

735980 

740984 

745046 

725662 

728140 

730290 

scr20 

110030 

0.69 

111420 

112420 

114334 

110030 

110030 

110030 

ste36a 

9526 

0.7 

9856 

10045 

10176 

9634 

9679 

9700 

ste36b 

15852 

0.7 

17118 

17280 

17532 

15852 

16063 

16378 

ste36c 

8239110 

0.7 

8300254 

84992800 

8586740 

8291532 

8308740 

8343504 

3 


2 


GRASP-D  time 
GRASP-S  time  1.5 


1 

.8 


0 0.2  0.4  0.6  0.8  1 

sparsity 


Figure  3.5:  Solution  time  ratios  as  a function  of  problem  sparsity  (2048  iterations). 
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Table  3.3;  Summary  of  runs  for  32  and  2048  iterations  ( 40  < n < 150) 

32  iterations  2048  iterations 


name 

bks 

Sm 

min 

avg 

max 

min 

avg 

max 

esc 128 

64 

0.98 

64 

64 

64 

64 

64 

64 

esc64a 

116 

0.95 

116 

116 

116 

116 

116 

116 

lipa40a 

31538 

0 

31933 

31949 

31961 

31884 

31902 

31911 

lipa40b 

476581 

0 

476581 

493113 

559240 

476581 

476581 

476581 

lipaSOa 

62093 

0 

62767 

62790 

62815 

62704 

62727 

62747 

lipaSOb 

1210244 

0 

1210244 

1339350 

1427509 

1210244 

1210244 

1210244 

lipaGOa 

107218 

0 

108211 

108243 

108262 

108077 

108148 

108192 

lipaGOb 

2520135 

0 

2520135 

2902810 

3000633 

2520135 

2520135 

2520135 

lipaTOa 

169755 

0 

171158 

171185 

171209 

171047 

171079 

171102 

lipa70b 

4603200 

0 

5516027 

5519179 

5522184 

4603200 

4603200 

4603200 

lipa80a 

253195 

0 

255066 

255123 

255176 

254861 

254941 

254976 

lipaQOa 

360630 

0 

363159 

363204 

363226 

362917 

362972 

363028 

lipaQOb 

12490441 

0 

15121756 

15136900 

15156071 

12490441 

12490441 

12490441 

skolOOa 

152002 

0.3 

153608 

153732 

153884 

152984 

153052 

153160 

skolOOb 

153890 

0.3 

155284 

155694 

156082 

154994 

155046 

155146 

skolOOc 

147862 

0.3 

149554 

149859 

150206 

148806 

148988 

149152 

skolOOd 

149576 

0.3 

151186 

151474 

151620 

150430 

150594 

150778 

skolOOe 

149150 

0.3 

150664 

150934 

151132 

150262 

150330 

150394 

skolOOf 

149036 

0.3 

150544 

150787 

151138 

150222 

150283 

150356 

sko42 

15812 

0.3 

16070 

16089 

16136 

15890 

15908 

15940 

sko49 

23386 

0.3 

23702 

23744 

23782 

23508 

23544 

23572 

sko56 

34458 

0.3 

34806 

34900 

34964 

34630 

34697 

34738 

sko64 

48498 

0.3 

49060 

49178 

49306 

48716 

48816 

48884 

sko72 

60402 

0.3 

66986 

67204 

67486 

66592 

66725 

66854 

sko81 

82277 

0.3 

91896 

92230 

92446 

91366 

91588 

91760 

sko90 

115534 

0.3 

116750 

117055 

117348 

116364 

116420 

116484 

thol50 

8134056 

0.57 

8239910 

8499280 

8586740 

8200926 

8205800 

8216528 

tho40 

240516 

0.58 

243682 

244979 

245608 

240650 

241870 

243088 

willOO 

273044 

0 

274254 

274585 

274918 

273964 

274127 

274326 

pure  quadratic  assignment  problems,  and  have  at  least  one  symmetric  distance  or 
flow  matrix. 

For  every  problem,  40  runs  were  performed  reflecting  eight  iteration  settings 
(16,  32,  64,  128,  256,  512,  1024  and  2048  iterations),  and  five  random  number 
generator  seeds  (1,  2,  3,  4,  5)  for  each  iteration  setting.  An  identical  computational 
run  for  GRASP-D  is  presented  in  Li  et  al.  [129],  so  a direct  comparison  can  be 
made  between  the  performances  of  GRASP-D  and  GRASP-S  in  terms  of  CPU 
time  and  solution  quality.  In  Tables  3. 1-3.3,  the  performance  of  the  GRASP-S  as 
reported  for  32  and  2048  iterations  is  presented,'  where  the  minimum,  average  and 
the  maximum  solution  value  is  shown  for  every  instance.  In  Figures  3. 4-3. 5,  the 
speedup  in  solution  time  as  a function  of  sparsity  is  presented,  where  speedup  is  the 
ratio  of  the  GRASP-D  time  over  GRASP-S  time.  As  it  can  be  seen  from  the  figures, 
for  instances  with  sparsity  > 0.8,  the  GRASP-S  procedure  is  on  average  twice  as 
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fast  as  GRASP-D.  Furthermore,  for  completely  dense  problems  (i.e.  sparsity  = 0) 
GRASP-S  and  GRASP-D  have  comparable  performances. 

3.6  Conclusion 

In  this  chapter  we  present  a modified  version  of  the  GRASP  for  the  QAP 
for  handling  instances  with  sparse  input  data.  GRASP-S  is  shown  to  be  about 
35  percent  faster  than  GRASP-D,  without  affecting  the  solution  quality  of  the 
heuristic.  Moreover,  for  very  sparse  instances  (i.e.  with  sparsity  greater  or  equal 
to  0.8),  the  sparse  version  performs  up  to  300  percent  faster,  indicating  that  large 
QAP  instances  with  at  least  one  sparse  matrix  can  be  solved  in  much  less  time. 


CHAPTER  4 

THE  TURBINE  BALANCING  PROBLEM 

4.1  Introduction 

Turbine  engines,  either  hydraulic,  gas  or  steam  powered,  are  composed  of  “fans” 
which  rotate  at  high  speeds  around  the  central  axis  of  the  turbine.  Each  fan  (see 
Figure  4.1)  is  composed  of  a number  of  blades  placed  around  the  periphery  of  a 
disk  at  equally  spaced  points.  The  masses  of  the  blades  are  not  identical  due  to 
errors  in  the  manufacturing  process  and  wear  through  use.  These  masses  can  vary 
as  much  as  ±5%  ([142]).  Therefore  the  center  of  mass  of  the  assembled  fan  depends 
on  the  positioning  of  the  blades  around  its  periphery.  When  the  center  of  gravity 
of  a fan  does  not  coincide  with  its  central  axis  of  rotation  then  static  unbalance 
occurs.  The  deviation  of  the  center  of  gravity  of  the  fan  from  the  axis  of  rotation 
is  referred  as  the  unbalance  of  the  turbine.  Static  unbalance  is  not  desirable  since 
it  results  in  vibrations  and  shortens  the  life  of  the  equipment. 

The  current  practice  in  balancing  turbines  is  to  find  a “good”  arrangement 
of  the  blades  based  on  a trial  and  error  process  and  intuition,  and  correct  any 
residual  unbalance  by  adding  weights.  This  process  is  time  consuming,  and  it  may 
take  even  weeks  depending  on  the  type  of  the  turbine  and  the  number  of  fans  it 
contains.  The  process  of  balancing  a turbine  has  to  be  performed  in  the  assembly 
line,  where  the  turbine  is  manufactured,  and  thereafter  every  time  it  is  maintained. 
Therefore  an  efficient  way  to  balance  a turbine  quickly  is  considered  to  be  very 
beneficial. 
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fans 
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central  axis  of  rotation 
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Turbine 


Figure  4.1:  A turbine  engine 


In  this  chapter  the  turbine  balancing  problem  (TBP)  is  formulated  as  a stan- 
dard QAP,  and  a modified  version  of  the  GRASP  heuristic  described  in  chapter 
2 is  presented,  for  solving  the  TBP.  Computational  results  indicate  that  the  pro- 
posed algorithm  outperforms  current  practice  and  it  is  competitive  with  respect 
to  previously  proposed  algorithms  for  solving  the  TBP  that  have  appeared  in  the 
literature. 

The  TBP  was  first  addressed  as  a discrete  combinatorial  optimization  problem 
by  Mosevich  [142]  in  1986.  In  Mosevich  [142]  a Monte  Carlo  approach  was  used 
for  finding  approximate  solutions  to  the  TBP.  Specifically  random  feasible  solu- 
tions were  generated,  and  the  one  with  the  least  objective  function  value  was  kept. 
Korenjak  and  Batagelj  [118]  present  a local  search  algorithm  for  solving  the  TBP, 
which  performed  a 2-opt  neighborhood  search  on  randomly  generated  initial  feasi- 
ble solutions.  They  also  introduced  a set  of  test  problems  for  testing  their  approach, 
which  will  be  referred  to  as  the  integer  turbine  balancing  problem  (ITBP),  and  will 
be  described  in  more  detail  in  §4.4.  Laporte  and  Mercure  [122]  formulated  the  TBP 
as  a QAP,  and  presented  a series  of  heuristics  for  solving  it.  Their  heuristics  are 
based  on  an  interchange  algorithm  by  Or  [148]  for  solving  the  traveling  salesman 
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problem,  and  the  computational  results  indicate  an  improvement  from  the  Monte 
Carlo  approach  used  by  Mosevich.  Fathi  and  Ginjupalli  [67]  proposed  two  types 
of  heuristics  for  solving  the  QAP  formulation  of  the  TBP,  and  also  present  some 
combinatorial  characteristics  of  the  ITBP.  The  first  family  of  heuristics  is  based 
on  a greedy  approach,  where  the  “heaviest”  blade  is  placed  in  a arbitrary  position 
and  the  subsequent  blades  are  placed  in  a greedy  fashion.  The  second  family  of 
heuristics  is  based  on  partitioning  the  set  of  blades  into  subsets  of  equal  cardinal- 
ity, and  can  only  be  applied  to  ITBPs  with  a prime  number  of  blades.  Mason  and 
Ronnqvist  [134]  model  the  TBP  as  a variation  of  the  standard  QAP,  and  propose  a 
number  of  local  search  algorithms  which  use  as  starting  points  solutions  obtained 
from  a Lagrangean  dual  scheme.  Amiouny  and  Bartholdi  [9]  present  a number  of 
heuristics  which  are  modifications  of  the  current  practice  used  for  balancing  turbine 
engines.  Although  their  heuristics  perform  worse  than  the  previously  mentioned 
approaches,  they  have  the  advantage  of  small  computational  times  and  worst-case 
guarantees  in  terms  of  residual  unbalance.  The  worst-case  bounds  they  provide 
are  based  in  terms  of  the  difference  between  successive  weights,  when  the  weights 
of  the  blades  are  sorted  in  descending  order. 

4.2  The  Turbine  Balancing  Problem 

In  this  section  it  will  be  shown  how  the  TBP  can  be  formulated  as  a min- 
imization problem  which  is  equivalent  to  a standard  QAP.  Before  we  formulate 
the  TBP  as  a minimization  problem  certain  assumptions  regarding  the  physical 
characteristics  of  the  blades  and  the  fan  as  a whole  need  to  be  made.  Specifically: 

• The  weights  of  the  blades  are  known.  There  are  various  methods  used  in  the 
industry  to  measure  the  blade  weights. 
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• The  center  of  gravity  for  each  blade  lies  at  a distance  r from  the  geometrical 
center  of  the  fan  and  in  the  same  plane,  when  the  blade  is  placed  on  the 
periphery  of  the  fan. 


• The  center  of  gravity  of  the  fan  (without  the  blades)  lies  in  its  geometrical 
center  and  it  has  zero  weight. 


A y 


♦ 

f 

Figure  4.2:  Fan  moment  vector  with  n blades 

The  above  assumptions  allow  us  to  consider  a two-dimensional  model  of  the  fan, 
where  the  blades  are  treated  as  point  masses.  It  is  noted  that  the  above  assump- 
tions are  in  accordance  with  the  current  method  used  in  the  industry  to  balance 
turbines. 

Consider  that  we  have  n blades  (see  Figure  4.2),  each  with  a weight  Wi,i  — 
1, . . . , n,  to  be  placed  in  n positions  where  each  position  is  described  by  a vector 
~ (rsin0,’)>  ~ ~ 1)/^)  i = 1, . . . ,n.  Let  Hat  be  the  set  of  all  permutations 
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of  the  set  of  integers  :=  {1, 2, . . . , n}.  For  a given  permutation  p (or  equivalently 
a positioning  of  blades  into  the  periphery  of  the  fan),  the  center  of  gravity  of  the 
fan  is  located  at 


n 

2^  WiV  cos 

j=l 


n 


6i,  2^  Wir  sin  9j 


:1 


In  the  TBP  we  want  to  find  a permutation  such  that  the  center  of  gravity  of  the 
fan  (4.1)  is  closest  to  the  origin.  Equivalently  we  want  to  find  a permutation  that 
minimizes  the  Euclidean  norm  of  the  moment  of  the  fan  about  its  geometrical 
center 


min 

peHjv 


where  for  a given  x € R”  the  Euclidean  norm  is  defined  as  ||x||  :=  \ZXliLi  ^1- 
blade  position  i in  Figure  4.2  is  represented  by  a position  vector  dj,  and  each  blade 
has  a given  mass  Wi.  Therefore  the  moment  vector  generated  by  placing  blade 
with  mass  Wi  in  the  position  described  by  the  vector  dj  is  given  by  irij  = Widi. 
The  moment  vector  of  the  whole  disk  containing  the  blades,  in  other  words  the 
fan  itself,  would  be  the  sum  of  the  individual  moment  vectors  of  the  blades.  The 
problem  is  to  assign  the  blades  in  such  a way  such  that  the  magnitude  of  the  sum  of 
the  individual  moment  vectors  is  minimized.  A large  fan  moment  vector  translates 
into  an  off-balance  turbine  which  results  in  vibrations  when  the  turbine  rotates 
about  its  central  axis. 

Given  two  symmetric  matrices  in  R"^",  F = (fij)  and  D = (dki)  with  nonzero 
diagonal,  the  standard  QAP  is  (see  §2.1) 


n n 


min 

PGIItv 


i=i  j=i 
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The  TBP  can  be  formulated  as  a standard  QAP  where  the  matrices  F = {fij)  and 
D = {dki)  are  formed  by  the  weights  and  the  angles  of  the  blade  positions. 

Remark  4.1  The  problem  in  (4-2)  is  equivalent  to  (4-3)  with  dij  = W{Wj,fij  = 
cos{6i  — 9j)  for  i,j  = l,2,...,n. 


Proof  4.1  Consider  the  problem  in  (4-3): 


mm 

penjv 


n 

2=1 

= min 

PGIIat 

n 


i=l 


r cos  9i 
r sin  04 


r mm 


n 


n 


cos  I + ^ Wp(i)  sin  6>, 


i=l 


i=l 


= r 


n n 


min  2^  ’^p(i)'^p{j)  (cos  9i  cos  9j  + sin  9{  sin  9j) 

2=1  J = 1 


= r‘ 


n n 


2=1  j = l 


-«i) 


Let  dij  = WiWj,  fij  = cos(0j  — 9j)  for  i,j  = 1, 2, . . . ,n  and  the  result  follows. 


The  turbine  balancing  problem  is  NP-Hard.  This  was  shown  by  Amiouny  [8] 
by  reduction  from  the  partition  problem  which  is  known  to  be  NP-complete.  A 
different  proof  by  reduction  from  the  even-odd  partition  is  also  given  by  Qela  [45]. 


4.3  GRASP  for  the  TBP 

In  this  section  a GRASP  procedure  for  solving  the  TBP  will  be  presented. 
Following  the  description  of  GRASP  in  section  §2.10.7,  the  construction  and  the 
local  search  phases  will  be  described,  which  are  the  main  steps  of  the  algorithm. 
The  following  discussion  is  also  related  to  that  of  §5.2.4. 

The  minimization  problem  under  consideration  is  stated  in  (4.3)  and  recall 
from  remark  4.1  that  dij  = WiWj  and  fij  = cos{9i  - 9j).  Therefore  F = (fij)  is  a 
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symmetric  matrix  with  ones  in  the  diagonal  and  D = (dij)  is  also  symmetric  with 
nonzero  diagonal.  The  construction  phase  consists  of  two  stages.  In  the  first  stage 
we  make  the  first  two  assignments  simultaneously,  by  choosing  a 2-permutation 
randomly  from  the  restricted  candidate  list  (RCL).  The  RCL  contains  tuples  of 
partial  permutations  and  values  associated  to  them.  Specifically  given  a pair  {p,  q) 
of  2-permutations  we  can  associate  with  it  the  value 


2 2 


i=l  j=l 

Clearly,  the  2-permutations  p,  q can  be  seen  as  elements  of  the  set  K = 
{(bi)  I hj  = 1)2,...  ,n,i  ^ j},  and  since  \K\  = n{n  — 1),  there  are  n^(n  — 1)^ 
pairs  (p,  q)  of  2-permutations.  Since  the  matrices  F and  D are  symmetric  and  F 
has  ones  in  the  diagonal,  the  above  cost  simplifies  to 


Cp^g  - 2dp(i)p(2)/,(l)9(2)  + dp(i)p(i)  + dp(2)p(2)- 

The  RCL  contains  a number  of  pairs  (p,  q)  - this  number  equals  the  RCL  size  and 
is  denoted  by  RCLsize  - having  the  smallest  associated  costs.  We  compute  the 
costs  Cp^q  for  all  n^(n  — 1)^  {p,q)  pairs  and  keep  the  RCLsize  smallest  among 
them.  Finally,  one  pair  of  2-permutations  from  RCL  is  chosen  at  random  at  stage 
one,  and  this  pair  determines  the  positions  of  two  blades  which  are  kept  fixed  in 
the  second  stage  of  the  construction  phase.  Notice  that  the  RCL  is  constructed 
only  once,  and  hence,  in  constant  time  with  respect  to  the  number  of  iterations. 

In  the  second  stage  the  remaining  n — 2 assignments  of  blades  to  positions  are 
made.  Let  the  set  F^  be  the  set  of  assignments  made  prior  to  the  r — th  assignment: 


{ (jl ) b ) ) (j*2)  ^2) ) • ■ • ) (jr— 1 ) b— 1 ) }" • 
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Note  that  at  the  start  of  stage  2,  jPal  = 2,  since  two  assignments  are  made  in 
the  first  stage,  and  r = iFrl  + 1 throughout  the  second  stage.  In  stage  2 we  also 
construct  an  RCL  which  contains  the  single  assignments  m — >■  s,  (m,  s)  ^ and 
their  associated  costs  Cms  defined  as 


Cms  • 2^  ^V{i)p(j)fq{i)q{3)i 

{i,j)eTr 


where 


Tr  ■=  {{i,j)  I i,j  = 1,2,...  ,r,  {r}  ^ 0} 


The  r-permutations  p,q  are  defined  as  p{i)  = ji,q{i)  = ij,  z = 1, . . . , r - 1 and 

p{r)  = m,  q{r)  = s.  Since  F and  D are  symmetric  the  cost  Cms  is  given  by  the 
simpler  formula 

r— 1 

Cms  2 ^ ^ dmp{i)  f sq{i)  T dmm-  (d-4) 

i=l 


Among  the  U = {n  — r + iy  possible  assignments  (m,  s),  those  with  the  RLSsize 
smallest  associated  costs  are  included  in  RCL.  One  assignment  is  then  selected  at 
random  from  RCL  and  the  set  F^  is  updated 


Fr  = Fr  U {(m,  s)}. 

This  process  is  repeated  until  a permutation  of  {1,2,...  ,n},  that  is,  a feasible 
solution  of  the  considered  QAP,  results.  Stage  2 of  the  construction  phase  of 
GRASP  in  pseudo-code  is  shown  in  Figure  4.3.  Procedure  ConstructionStage2 
returns  the  set  F with  n — 1 assignments,  the  last  assignment  being  then  trivial. 
The  inheap  0 and  outheapO  procedures  are  used  for  sorting  and  choosing  the 
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procedure  ConstructionStage2(a,  {ji,pi),  {j2,P2)) 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 
17 


ra  = {{juPi)Ah,P2)}] 

do  T — 3,  . . . 5 — 1 — y 

[7  = 0; 

do  m = 1, . . . , n — > 
do  s = 1, . . . ,n 
if  (m,  s)  ^ Fr 

Cms  2 f + d 

inheap(C^s); 
t/  = C/+l; 

fi; 

od; 
od; 

t =random[l,  [oJ/J]; 
do  2 — Ij . , , ^ 

Cms  =outheap(); 
od; 

Fr+i  = Fr  U {(m,  s)}; 


mm’t 


18  od; 

19  return  F 

end  Construct ionStage2; 


Figure  4.3:  Stage  2 of  Construction  Phase  of  GRASP 

smallest  among  the  computed  Cm,s  costs,  respectively.  The  procedure  random 
generates  a random  number  in  the  interval  [1,  [ot/J].  The  parameter  a ranges 
between  1 and  0.  When  a is  close  to  0 the  algorithm  is  more  greedy,  while  when 
it  is  close  to  1 the  algorithm  is  more  random. 

Finally,  the  second  phase  of  the  algorithm  completes  a GRASP  iteration  by 
applying  an  improvement  method  starting  from  the  solution  constructed  in  the 
first  phase  and  employing  the  2-exchange  neighborhood  (see  also  §2.10.3).  Given 
two  permutations  p and  q that  differ  in  only  two  positions, 


p{ko)  = q(lo)  and  p{lo)  = qiK), 
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we  can  compute  their  difference  in  the  objective  function  cost  in  0{n)  time  by 
evaluating  the  summation 

/ ^ fij{^p(i)pU)  ~ ^9(090'))’ 

where 

K = {(b  j)  I = 1, 2, . . . , n,  {i,  j}  n {ko,  lo}  ^ 0}, 

and 

iA^I=2‘(j)(n-2)‘  + 2^(^)(n-2)«. 

4.4  Computational  Results 

Two  sets  of  problems  were  used  for  testing  the  performance  of  the  proposed 
algorithm.  The  first  set  was  provided  by  Brian  Center,  an  engineer  at  Pratt  & 
Whitney  (P&W)  which  is  a major  turbine  engine  manufacturer.  This  problem  set 
contains  data  for  the  NASA  Space  Shuttle  Main  Engine  Advanced  Turbopump 
High  Pressure  Oxygen  rotor.  Each  turbine  contains  5 discs  (fans)  each  carrying 
54  blades.  The  results  are  shown  in  table  4.1.  The  first  column  contains  the  name 
of  the  turbine.  The  second  column  contains  the  disc  number  for  each  turbine,  and 
the  fourth  column  the  radius  r of  the  disc.  The  solution  obtained  from  the  current 
method  employed  in  P&W,  and  that  obtained  by  GRASP  are  shown  in  columns 
5 and  7 respectively.  For  turbine  P022-02  the  solution  obtained  by  PWBAL,  a 
commercial  turbine  balancing  software  distributed  by  P&W,  is  given  in  column  6. 
In  column  8 the  number  of  iterations  needed  by  GRASP  to  reach  the  given  solution 
is  reported.  The  maximum  number  of  iterations  that  GRASP  was  allowed  to  run 
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for  all  the  problems  was  1000,  which  required  approximately  11  CPU  minutes  on 
an  Intel  Pentium  200MHz  processor  with  64MBytes  of  memory.  A PC  was  used  for 
performing  the  computational  experiments,  since  the  balancing  of  turbines  usually 
occurs  in  the  assembly  floor  where  PC’s  are  available.  Comparison  between  the 
solution  values  of  GRASP  and  the  current  method  used  at  P&W  demonstrates 
that  GRASP  outperforms  by  far  the  current  method.  On  average  the  solutions 
provided  by  GRASP  are  3 orders  of  magnitude  smaller  than  those  provided  by  the 
current  method.  In  addition  to  the  solution  quality,  GRASP  is  also  much  faster 
since  it  requires  approximately  one  hour  for  the  whole  turbine,  while  the  current 
method  might  require  a week  to  complete  the  balancing  process.  Therefore  an 
implementation  of  GRASP  in  the  assembly  floor  would  translate  in  large  savings  in 
production  time,  and  increased  quality.  Moreover,  GRASP  also  compares  favorably 
with  the  commercial  package  PWBAL.  All  solutions  provided  by  GRASP  for  the 
P022-02  turbine  are  less  than  the  ones  provided  by  PWBAL.  The  second  set  of 


Table  4.1:  Statistics  summarizing  GRASP  runs  for  the  P&W  problem  set 


Turbine 

disc 

n 

r 

current 

PWBAL 

GRASP 

iter. 

P022-02 

1 

54 

4.5286 

0.0059 

6.40  X 10-« 

5.16  X 10-“ 

874 

2 

54 

4.2725 

0.0057 

9.80  X 10-® 

3.50  X 10-® 

91 

3 

54 

4.5640 

0.0165 

2.93  X 10-® 

3.67  X 10-® 

416 

4 

54 

4.2725 

0.0022 

9.80  X 10-® 

4.69  X 10-® 

276 

5 

54 

4.5927 

0.0040 

2.32  X 10~^ 

4.94  X 10~® 

40 

P022-01 

1 

54 

4.5286 

0.0006 

6.29  X 10-“ 

116 

2 

54 

4.2725 

0.0005 

7.07  X 10-® 

652 

3 

54 

4.5640 

0.0164 

1.08  X 10-5 

616 

4 

54 

4.2725 

0.0044 

8.65  X 10-® 

925 

5 

54 

4.5927 

0.0064 

1.07  X 10-5 

103 

P024-01 

1 

54 

4.5286 

0.0086 

1.75  X 10-5 

763 

2 

54 

4.2725 

0.0043 

5.28  X 10-5 

175 

3 

54 

4.5640 

0.0048 

9.09  X 10-5 

97 

4 

54 

4.2725 

0.0005 

9.97  X 10-5 

201 

5 

54 

4.5927 

0.0064 

1.15  X 10-5 

904 

P030-01 

1 

54 

4.5286 

0.0039 

5.95  X 10-5 

995 

2 

54 

4.2725 

0.0002 

1.30  X 10-5 

165 

3 

54 

4.5640 

0.0043 

1.00  X 10-5 

185 

4 

54 

4.2725 

0.0022 

1.35  X 10-5 

395 

5 

54 

4.5927 

0.0121 

4.94  X 10-5 

773 
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problems  used,  comes  from  the  literature  and  do  not  correspond  to  actual  data  of 
turbine  engines.  Introduced  by  Korenjak  and  Batagelj  [118]  it  is  called  the  integer 
turbine  balancing  problem  (ITBP)  problem  set,  and  we  choose  to  include  it  in 
our  computational  results  since  it  will  illustrate  how  GRASP  compares  to  other 
algorithms  for  solving  the  TBP.  An  instance  of  ITBP  of  size  n > 2 is  generated 
by  setting  Wi  = i,i  = 1,...  ,n.  Note  that  by  the  definition  of  the  TBP,  the 
angles  0i,i  = 1, . . . ,n  are  the  same  for  every  instance  of  size  n since  the  blades  are 
placed  at  equally  spaced  points.  The  computational  results  for  the  ITBP  problem 
set  are  shown  in  table  4.2.  The  first  column  corresponds  to  the  size  n of  the 
problem,  while  the  radius  of  the  disc  for  every  problem  was  set  to  r = 1000.  The 
second  column  corresponds  to  the  solution  provided  by  the  algorithm  of  Korenjak 
and  Batagelj  [118]  which  we  call  KBH.  Columns  3 through  8 report  the  solution 
given  by  the  algorithms  by  Fathi  and  Ginjupalli  [67].  They  propose  six  heuristic 
methods  based  on  two  general  schemes,  the  placement  heuristic  (PHA:,  k = 1, 2, 3) 
and  the  rotational  heuristic  (RHA:,  k = 2,3,4).  The  solution  found  by  GRASP 
and  the  number  of  iterations  needed  are  given  in  columns  9 and  10  respectively. 
The  maximum  number  of  iterations  was  set  at  10000.  In  column  11  the  CPU 
seconds  needed  for  GRASP  to  complete  10000  iterations,  or  find  a solution  which 
results  in  0 unbalance  (the  theoretical  lower  bound)  is  given.  A zero  value  of  CPU 
seconds  means  that  the  CPU  time  was  less  than  a second.  The  rotational  heuristic 
RHA;  only  works  for  ITBPs  with  n divisible  by  k,  while  Korenjak  and  Batagelj 
[118]  report  solutions  for  ITPB  instances  with  n < 20.  From  table  4.2  we  can 
see  that  GRASP  clearly  outperforms  the  heuristics  PHA:,  k = 1,2,3.  The  KBH 
heuristic  produces  the  same  solutions  as  GRASP  for  instances  with  n < 12,  but  as 
n gets  larger  its  solutions  deteriorate  with  respect  to  the  ones  given  by  GRASP. 
In  comparing  the  RHA:  heuristics  with  GRASP  we  have  to  mention  the  following 
result  regarding  a combinatorial  characteristic  of  the  ITBP. 


133 


Lemma  4.1  (Fathi  and  Ginjupalli  [67],  1992) 

An  ITBP  of  size  n has  a perfect  configuration  ifn  is  divisible  by  2 but  not  divisible 
by  4,  or  if  n is  divisible  by  3 but  not  divisible  by  9. 

The  authors  in  Fathi  et  al.  [67]  further  conjecture  that  an  ITBP  of  size  n has  a 
perfect  configuration  if  n is  divisible  by  k but  not  divisible  by  for  any  prime 
number  k > 2.  A perfect  configuration  is  an  assignment  of  blades  into  positions 
that  results  in  0 unbalance.  The  RHA;  heuristics  justify  the  above  conjecture,  but 
perform  poorly  for  values  of  n that  do  not  satisfy  the  above  condition.  GRASP 
finds  a perfect  configuration  for  all  the  values  of  n that  satisfy  the  above  conditions, 
except  for  n = 15, 20.  Overall  GRASP  performs  better  than  all  of  the  heuristics 
mentioned  and  it  also  provides  best  known  values  for  the  ITBP  instances  with 
n = 13, 17, 19,23.  The  RRk  heuristic  seems  to  be  the  only  competitor,  but  it  can 
only  be  applied  to  ITBPs  with  n divisible  by  k,  and  it  performs  rather  poorly  if  n 
does  not  satisfy  the  conditions  stated  in  lemma  4.1  and  the  related  conjecture. 

4.5  Conclusions 

The  turbine  balancing  problem  is  a combinatorial  optimization  problem  arising 
in  the  manufacturing  and  maintenance  of  turbine  engines.  Exact  solution  proce- 
dures for  solving  the  TBP  would  not  be  appropriate,  since  the  problem  is  NP-Hard 
and  has  to  be  solved  in  real  time  in  the  assembly  floor.  Furthermore  the  input  data 
is  not  accurate  due  to  errors  occurring  in  measuring  the  weights  of  the  blades,  and 
the  required  assumptions  made  for  modeling  the  problem.  Therefore  heuristic  pro- 
cedures which  can  solve  the  TBP  with  minimal  computational  effort,  and  provide 
solutions  that  fall  within  some  tolerance  level  acceptable  by  the  manufacturer,  are 
desirable. 

In  this  chapter  the  TBP  is  formulated  as  a QAP,  and  a heuristic  algorithm  is 
proposed  for  solving  the  resulting  problem.  The  proposed  algorithm  outperforms 
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Table  4.2:  Statistics  summarizing  GRASP  runs  for  the  ITBP  problem  set 


n 

KBH 

PHI 

Fathi  and  Ginjupalli 
PH2  PH3  RH2  RH3 

RH4 

soln 

GRASP 

iter.  CPU  sec. 

5 

29.93 

29.93 

29.93 

29.93 

29.93 

1 

1 

6 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

3 

0 

7 

2.73 

2.73 

2.73 

2.73 

2.73' 

2 

7 

8 

3.64 

30.06 

3.64 

3.64 

30.06 

30.06 

3.64 

24 

10 

9 

0.63 

13.94 

0.63 

0.63 

20.48 

0.63 

6 

16 

10 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1 

0 

11 

0.02 

2.09 

2.09 

0.89 

0.02 

1335 

35 

12 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

46 

9 

13 

0.04 

8.00 

2.74 

1.79 

0.03 

3438 

58 

14 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1019 

40 

15 

0.26 

15.68 

0.84 

0.38 

0.00 

0.01 

8431 

107 

16 

0.02 

7.49 

0.54 

0.23 

0.37 

3.10 

0.02 

3375 

143 

17 

0.05 

5.03 

0.14 

0.11 

0.03 

3575 

154 

18 

0.03 

4.06 

0.99 

0.92 

0.00 

0.00 

0.00 

4037 

77 

19 

0.05 

0.73 

0.72 

0.40 

0.01 

1799 

257 

20 

0.04 

1.94 

0.30 

0.00 

0.00 

0.00 

0.01 

1765 

296 

21 

3.05 

0.82 

0.75 

0.00 

0.00 

624 

20 

22 

2.95 

0.74 

0.56 

0.00 

0.00 

3427 

141 

23 

1.63 

0.31 

0.13 

0.00 

9848 

465 

24 

2.46 

0.87 

0.49 

0.00 

0.00 

0.00 

0.00 

203 
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the  current  practice  as  indicated  by  computational  results  on  a set  of  problems 
provided  by  Pratt  & Whitney.  Moreover  the  algorithm  has  the  best  overall  per- 
formance with  respect  to  other  heuristic  algorithms  taken  from  the  literature. 


CHAPTER  5 

BIQUADRATIC  ASSIGNMENT  PROBLEM 


5.1  Introduction 

The  biquadratic  assignment  problem  (BiQAP)  is  a generalization  of  the 
quadratic  assignment  problem  (QAP).  It  is  a nonlinear  integer  programming  prob- 
lem where  the  objective  function  is  a fourth  degree  multi-variable  polynomial  and 
the  feasible  domain  is  the  assignment  polytope.  Motivated  by  a practical  prob- 
lem arising  in  VLSI  synthesis,  the  problem  was  first  introduced  and  studied  by 
Burkard,  Qela  and  Klinz  [41]. 

Two  formulations  for  the  BiQAP  were  introduced  in  §2.7.1,  and  here  we  restate 
the  permutation  formulation 


min 


n n n n 
i-l  j=l  k=\  1=1 


where  H;v  denotes  the  set  of  all  permutations  of  N = {1,2,...  , n}.  This  formula- 
tion of  the  BiQAP  will  be  used  throughout  this  chapter. 

Burkard,  Qela  and  Klinz  [41]  showed  that  the  BiQAP  is  NP-hard.  They  com- 
puted lower  bounds  for  BiQAP  derived  from  lower  bounds  of  the  QAP. 

In  this  chapter,  we  describe  a greedy  randomized  adaptive  search  procedure 
(GRASP)  for  the  BiQAP  and  examine  its  performance  on  the  set  of  BiQAP  in- 
stances considered  in  Burkard  et  al.  [40].  A GRASP  is  an  iterative  randomized 
meta-heuristic  that  has  been  applied  to  many  combinatorial  optimization  prob- 
lems, including  the  QAP  (see  [130,  154,  153,  129]). 
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The  remainder  of  the  chapter  is  organized  as  follows.  In  §5.2,  we  describe 
the  GRASP  for  BiQAP  and  discuss  the  relation  of  this  heuristic  to  GRASPs  for 
other  nonlinear  assignment  problems.  Experimental  results  are  presented  in  §5.3. 
Concluding  remarks  are  made  in  §5.4. 

5.2  GRASP  for  the  BiQAP 

A brief  introduction  of  GRASP  is  given  in  §2.10.7  GRASP  has  been  imple- 
mented to  solve  QAPs  with  dense  [130, 129]  and  sparse  coefficient  matrices  [154].  A 
parallel  implementation  of  a GRASP  for  QAP  is  described  in  Pardalos  et  al.  [153]. 

The  feasible  space  for  a BiQAP  of  size  n is  the  same  as  for  a QAP  of  that  size, 
that  is,  the  set  of  all  possible  n!  permutations  of  the  set  N = {1,2,...  , n}.  Each  of 
the  n components  of  the  permutation  vector  corresponds  to  an  assignment.  During 
the  GRASP  construction  phase  a complete  permutation  is  constructed.  Initially, 
four  assignments  are  made  simultaneously.  After  that,  one  assignment  is  made  at  a 
time.  Each  new  assignment  added  to  a partial  permutation  contributes  to  the  total 
cost  of  the  assignment  the  cost  of  its  interaction  with  already-made  assignments. 
To  construct  the  permutation,  costs  are  computed  for  each  feasible  assignment  and 
among  those  with  small  cost,  one  is  randomly  selected  to  be  added  to  the  partial 
permutation.  As  is  the  case  in  the  GRASP  for  QAP,  the  construction  phase  of 
the  GRASP  for  BiQAP  has  two  stages.  The  first  stage  makes  the  four  initial 
assignments  simultaneously,  while  the  second  stage  makes  the  remaining  n — 4, 
one  assignment  at  a time.  In  the  GRASP  local  search  phase,  2-exchange  local 
search  is  applied  to  the  permutation  constructed  in  the  first  phase. 

In  the  remainder  of  this  section,  we  provide  a detailed  description  of  the  com- 
ponents of  the  GRASP  for  the  BiQAP. 
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5.2.1  Stage  1 of  the  Construction  Phase 

Stage  1 of  the  GRASP  construction  phase  makes  the  first  four  assignments 
simultaneously.  For  these  assignments,  the  greedy  choice  is  the  set  of  four  as- 
signments with  the  minimum  cost  of  interaction.  Note  that  the  minimum  feasible 
number  of  initial  assignments  is  four,  since  the  BiQAP  is  only  defined  for  problems 
with  dimension  n > 4 and  costs  can  only  be  computed  if  at  least  four  assignments 
are  made. 

To  select  the  four  assignments,  the  interaction  cost  of  each  set  of  four  assign- 
ments must  be  computed.  The  sets  of  assignments  with  small  costs  are  placed  in 
a restricted  candidate  list  (RCL)  and  one  set  is  selected  at  random  from  the  RCL. 
Consider  the  computation  of  the  interaction  cost  of  four  assignments  in  a BiQAP 
instance  with  input  coefficient  arrays  A = (ajjki)  and  B = (bmpst)  of  elements 
each.  More  specifically,  the  interaction  cost  of  the  set  A of  four  assignments  is 

4 4 4 4 

^-4  “ ^PA(i)PA{j)PAWPA{l)^QA(i)<lA{j)QA(k)qA{l)^  (5-1) 

i=l  j=l  k=l  1=1 

where  the  injections  p^,  Qa  ^ire  defined  as  the  quadruples: 

PAi  Qa  ^ • {(b  3t  ^1 0 I b ^ — 1)  2, . . . , H,  i A 3i  ^1  J ^ k A • 

The  number  of  feasible  sets  of  four  assignments  is  \K\^,  and  it  is  easily  seen  that 
\K\  = n(n  — l)(n  — 2)(n  — 3).  The  RCL  contains  all  sets  A of  four  assignments 
that  have  small  interaction  costs  Ca-  This  RCL  for  Stage  1 needs  only  to  be  setup 
once,  since  it  does  not  change  from  one  GRASP  iteration  to  the  next. 

The  following  procedure  is  used  to  create  the  RCL.  Let  a, /?  (0  < a, /3  < 1) 
be  the  RCL  parameters,  [xj  denote  the  largest  integer  smaller  or  equal  to  x,  and 
A(1  < A < l/fp)  be  the  maximum  number  of  generated  cost  parameters.  Since 
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there  are  O(n^)  sets  of  four  assignments,  generation  of  all  sets  is  time  consuming. 
Instead,  the  algorithm  generates  A random  permutation  pairs,  and  for  each  pair 
Pa  and  it  computes  the  associated  interaction  cost  Ca-  The  costs  are  sorted 
in  increasing  order,  and  the  [/3AJ  smallest  elements  are  considered  as  potential 
candidates.  The  RCL  is  made  up  of  the  [a/^AJ  smallest  elements.  Note  again 
that  no  change  will  be  made  in  the  order  of  these  elements  during  the  GRASP 
iterations,  and  therefore  this  phase  of  the  GRASP  can  be  performed  in  constant 
time,  since  it  simply  consists  of  selecting,  at  random,  an  element  from  a set.  The 
procedure  described  above  is  implemented  efficiently  using  heap  sort. 

5.2.2  Stage  2 of  the  Construction  Phase 

In  Stage  2 of  the  GRASP  construction  phase,  the  final  n — 4 assignments  are 
made,  one  assignment  at  a time.  Define  the  set  Pr  as  the  set  of  already-made 
assignments  prior  to  making  the  r-th  assignment  {r  > 5): 

{ (A ) Pi)  j (j*2 ) P2) ) * * • ) (j'r— 1 ? Pr— 1)  }” ■ 

At  the  start  of  Stage  2,  jPal  = 4,  since  four  assignments  are  made  in  Stage  1 of  the 
construction  phase.  Throughout  the  stage,  r = |Pr|  -I- 1. 

The  U = {n  — r + ly  possible  assignments  of  m to  s need  to  be  examined  to 
include  in  the  RCL  only  those  with  small  cost  with  respect  to  the  assignments  in 
set  Pr.  One  assignment  in  the  RCL  will  be  selected  at  random.  One  trivial  way  to 
determine  the  cost  of  assigning  m to  s is  to  compute  the  sum 

Cms  = E ^p(0p0‘)p(^)p(0^9(09(j)9(^)9(0’  (5-2) 
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where 


T :=  I = 1,2,. 


P{r)  = "i,  and  9(r)  = s.  One  can  easily  check  that  |T|  = r^,  so  to  compute  (5.2) 
requires  0{r*)  time.  A more  efficient  way  is  to  compute  the  sum 

^ms  E ^p{i)p{i)p{k)p{l)bq{i)q{j)q{k)q{l)  > (5-3) 

{i,j,k,l)eT' 


where 


T'  ■■=  {{i,j,k,l)  \i,j,k, 1 = 1,2,...  ,r,  and  {^,i, A:, /}  0 {r}  7^  0}, 

pp)  = m,  and  q^r)  = s.  Since  \T'\  = (J)(r  - 1)^  + (^(r  - 1)^  + (3)(r  - 1)  + 1, 
then  each  sum  can  be  calculated  in  0{r^)  time.  V contains  the  quadruples 
of  all  combinations  of  indices  that  correspond  to  the  order  of  the  already-made 
assignments,  having  at  least  one  occurrence  of  the  index  r.  Note  that  C' , < Cmi 
and  the  difference  Cms  — C'.^g  is  the  same  for  all  possible  assignments  m s,  which 
implies  that  the  smallest  corresponds  to  the  smallest  Cms- 

The  greedy  function  in  Stage  2 selects  the  assignment  that  has  the  minimum 
cost  of  interaction  with  respect  to  the  already-made  assignments,  that  is,  the  as- 
signment m to  s that  has  the  smallest  sum  Cms-  Consider  a GRASP  iteration  and 
let  a be  the  RCL  parameter  defined  in  Stage  1 of  the  construction  phase.  The  U 
costs  are  sorted  and  the  smallest  [cxU\  are  placed  in  RCL.  One  assignment  (say  m 
to  s)  is  selected  at  random  from  the  RCL  and  the  set  is  updated. 


Pr  = Tr  U {(m,  s)}. 
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procedure  ConstructionStage2(o,  {j2,P2),  (ia.Ps),  {u,P4)) 

1 r = {(ii,pi),  {j2,P2),  (is, Pa),  {u,P4)}l 

2 dor  = 5,  — 1 —¥ 

3 C/  = 0; 

4 do  m = 1, . . . , n — >■ 

5 do  5 — Ij . . , , u — ^ 

6 if  (m,  s)  ^ r — > 

^ C'ms  — ^(iJ^k,l)eT'  '^P{i)p(j)p{k)p{l)^q(i)q{j)q(k)q(l)  '> 

8 inheap  ; 

9 [/  = [/  + !; 

10  fi; 

11  od; 

12  od; 

13  t =random[l,  [of/J]; 

14  do  z — 1, , . . jt  — ^ 

15  =outheap() ; 

16  od; 

17  r = r U {{m,  s)} ; 

18  od; 

19  return  F 

end  ConstructStage2; 


Figure  5.1:  Pseudo-code  for  Stage  2 of  GRASP  construction  phase 

Stage  2 of  the  construction  phase  is  presented  in  pseudo-code  in  Figure  5.1. 

Procedure  ConstructStage2  takes  as  input  the  RCL  parameter  a and  the  four 
initial  assignments  made  in  Stage  1 and  makes  the  remaining  n — 4 assignments. 
In  lines  4 to  12,  the  costs  (Cms)  of  all  unassigned  pairs  are  computed  and  inserted 
into  the  heap  to  be  sorted.  Procedure  inheap  insert  an  element  into  the  head 
and  updates  the  heap.  In  lines  13  to  16,  a number  t is  randomly  generated  in 
the  interval  [1,  [aU\]  and  the  t-th  smallest  cost  is  retrieved  from  the  heap.  The 
procedure  outheap  deletes  the  smallest  element  from  the  heap  and  updates  the 
heap.  In  line  17  the  set  P of  already-made  assignments  is  updated.  The  algorithm 
continues  its  iterations  until  |P|  = n — 1,  in  which  case  the  solution  is  completed 
by  adding  the  n-th  remaining  assignment. 
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5.2.3  Local  Search  Phase 

The  second  phase  of  each  GRASP  iteration  is  local  search.  Given  a neighbor- 
hood definition,  the  neighborhood  N{s)  of  a solution  s is  searched  in  an  attempt 
to  identify  a better  solution  s'.  If  a better  solution  s'  exists  in  N{s),  the  current 

solution  is  replaced  by  s'.  The  procedure  is  repeated  until  no  further  improvement 
is  possible. 

In  the  implementation  described  in  this  chapter,  the  local  search  implemented 
is  the  2-exchange,  similar  to  the  local  search  for  the  QAP  (see  [130,  154,  153,  129]). 
We  next  describe  A;-exchange,  a generalization  of  2-exchange. 

Let  the  difference  between  two  permutations  p and  q be  S{p,q)  :=  {i  \ p{i) 
q{i)},  and  define  the  distance  between  the  two  permutations  to  be  d{p,q)  := 
\5{p,q)\.  The  A;-exchange  neighborhood  Nk{p)  for  a permutation  p G IlAr  is 


Nk{p)  :=  {q  I d{p,  q)  < k,2  < k <n}. 

Pseudo-code  for  the  /c-exchange  local  search  is  shown  in  Figure  5.2. 

The  size  of  the  neighborhood  used  in  the  A:-exchange  local  search  is  (")  = 
Let  p be  the  starting  permutation.  The  algorithm  examines  the  set  of 
permutations  obtained  by  interchanging  k elements  of  p.  In  line  3 of  the  pseudo- 
code, procedure  ProducePermutation(q',  m,  A^jt(p))  generates  a permutation  q by 
exchanging  k elements  of  the  current  permutation  p.  Which  elements  are  selected 
for  exchange  is  a function  of  the  value  of  m where  1 < m < (") . Once  a neighbor- 
ing permutation  that  improves  the  objective  function  is  found  (line  6),  the  current 
solution  is  updated  and  the  procedure  is  restarted,  searching  the  new  neighbor- 
hood. This  is  repeated  until  no  further  improvement  is  possible.  This  type  of 
search  is  called  first  decrement  search.  Another  type,  called  complete  enumera- 
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procedure  kexchcinge(A:,p) 
1 
2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 

end  kexchange; 


Figure  5.2:  Pseudo-code  of  A:-exchange  local  search 

tion,  examines  all  possible  solutions  that  can  be  generated  by  the  exchange  of  k 
elements,  that  is,  examines  the  complete  set  of  (^)  exchanges  in  the  neighborhood 
and  picks  the  best. 

In  our  implementation,  we  choose  first  decrement  2-exchange  search.  The 
neighborhood  A^2(p)  of  a fixed  permutation  p consists  of  all  those  permutations 
that  differ  from  p in  exactly  two  places,  that  is,  that  can  be  obtained  by  applying 
a transposition  to  p. 

As  we  see  in  line  6 of  the  pseudo-code,  the  calculation  of  the  objective  function 
value  of  a permutation  q G N{p)  where  p is  the  current  permutation,  is  repeated 
at  each  iteration  of  the  local  search.  In  the  worst  case,  there  are  (")  repetitions, 
that  is,  as  many  as  the  size  of  the  neighborhood.  Since  there  may  be  numerous 
repetitions,  it  is  desirable  to  have  an  efficient  procedure  to  decide  if  permutation 
q is  better  than  permutation  p.  Denote  by  Z{q)  the  objective  function  value  of 
a permutation  q.  A straightforward  evaluation  of  the  gain  AZ  = Z{p)  — Z{q) 
requires  O(n^)  time,  since  the  computation  of  Z{q),  for  any  q 6 IIat,  takes  0{n^) 
time.  However,  as  shown  in  Burkard  et  al.  [40],  AZ  may  be  computed  in  0{n^) 


StopFlag=false; 
do  StopFlag=f alse 
StopFlag=true; 
do  ui  = 1, . . . , — > 

ProducePermutation(^,  m,  Nk{p))] 
if  cost(^)  < cost  (p)  — >• 

p = q; 

StopFlag=f alse; 
break; 

fi; 

od; 

od; 

return  p 
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time,  using  the  following  procedure.  Suppose  that  d{p,  q)  = 2, 

p{K)  = q{lo)  and  p{lo)  = q{ko). 

To  compute  the  gain  AZ  = Z{p)  - Z{q),  we  evaluate  the  sum 

<^ijkl{i>p(i)p{j)p{k)p{l)  — bq(i)q(j)q{k)q{l)) 

(i,j,k,l)eK2 

where 

K?  ■■=  I ij,k,l  = 1,2,...  ,n,{i,j,k,l}n{koJo}  7^  0}- 

Since  \K2\  = 2^(^)(n  - 2)^  + 2^(2)(n  - 2)^  + 2^(g)(n  - 2)^  + - 2)°,  then  it 

follows  that  the  gain  can  be  calculated  in  O(n^)  time. 

5.2.4  Relating  GRASP  for  the  QAP,  BiQAP,  and  M-adic  Assignment 
Problem 

As  aforementioned,  the  QAP  is  a special  case  of  the  BiQAP,  and  it  is  natural  to 
expect  that  the  general  principles  of  GRASP  as  described  in  this  chapter  could  be 
applied  to  the  QAP.  The  reverse  however  is  not  true.  That  is,  the  implementation 
of  GRASP  for  the  QAP  as  described  in  Li  et  al.  [130]  cannot  be  directly  extended 
for  the  BiQAP,  because  it  is  so  designed  to  exploit  the  symmetry  of  the  matrices 
that  compose  a QAP  instance  (non-symmetric  QAPs  can  not  be  treated  in  [130]). 
The  implementation  of  GRASP  for  the  BiQAP  as  described  here  therefore  can  be 
thought  as  a generalization  of  the  heuristic  for  general  M-adic  assignment  prob- 
lems, and  in  this  section  we  will  show  that  this  implementation  reduces  to  the 
more  specialized  described  in  Li  et  al.  [130]  if  we  consider  symmetric  QAPs. 
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If  we  consider  Stage  1 of  the  construction  phase  as  applied  to  a QAP  of  size 
n,  we  must  make  the  two  initial  assignments  from  the  available  n,  each  with  an 
interaction  cost 


as  defined  in  (5.1),  where  a and  b are  n x n arrays  and  the  permutations  p^,  € 

^ — {{hj)  I bi  = 1)  2, . . . ,n,i  ^ j}.  Note  that  \K\  = n{n  — 1)  and  the  number  of 
feasible  sets  of  two  assignments  is  \K\^.  However,  for  a symmetric  QAP,  the  cost 
for  each  pair  of  assignments  reduces  to 


One  way  to  compute  the  sum  is  to  sort  the  non-diagonal  elements  of  a in 
increasing  order  and  of  b in  decreasing  order,  and  then  sort  their  corresponding 
products  in  increasing  order.  This  will  produce  a set  of  the  smallest  Ca  values 
that  constitutes  the  RCL,  exactly  as  defined  in  Li  et  al.  [130]. 

For  Stage  2 of  the  construction  phase,  following  the  definition  in  (5.3),  the  cost 
of  making  the  r-th  assignment  (of  m to  s)  for  a QAP  is 


2 2 


i=i  j=i 


Ca  — 20'p^(l)p^(2)5q_4(i)9_4(2)- 


(5.4) 


where 


145 


Considering  the  fact  that  the  QAP  is  symmetric  and  that  p{r)  = m and  q{r)  = s, 
the  cost  in  (5.4)  can  be  rewritten  as 


r— 1 

^rnp{j)t>sq{j), 

i=l 


which  is  equivalent  to  the  corresponding  cost  described  in  Li  et  al.  [130],  computed 
however  in  a more  efficient  way. 

Finally,  the  local  search  procedure  used  in  Li  et  al.  [130]  is  a 2-exchange  neigh- 
borhood search  where  the  gain  from  exchanging  ko  with  in  a given  solution  is 
computed  in  0{n)  time.  More  specifically,  given  two  permutations  p and  q that 
differ  in  only  two  positions. 


P{ko)  = q{lo)  and  p{lo)  = q{ko), 

we  can  compute  their  difference  in  the  objective  function  cost  by  evaluating  the 
summation 


<^ij{bp{i)p(j)  bq(i)q{j)) 


5 


{i,j)€K2 


where 


K2  = {{i,j)  I = 1,2,...  ,n,{i,j}n{ko,lo}  ^ 0}, 


and 


\K,\  = 2'Q{n-2)'  + 2^(^^{n-2r. 
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12  150  MHZ  IP19  Processors 

CPU:  MIPS  R4400  Processor  Chip  Revision:  5.0 

FPU:  MIPS  R4010  Floating  Point  Chip  Revision:  0.0 

Data  cache  size:  16  Kbytes 

Instruction  cache  size:  16  Kbytes 

Secondary  unified  instruction/data  cache  size:  1 

Mbyte 

Main  memory  size:  2560  Mbytes,  2-way  interleaved 


Figure  5.3:  Hardware  configuration  (partial  output  of  system  command  hinv) 

In  general,  for  an  M-adic  assignment  problem  the  cardinality  of  K2  which  defines 
the  complexity  of  performing  a 2-exchange  local  search  is 

2=1  ^ ' 

Considering  the  above  analysis,  it  should  be  clear  to  the  reader  how  the  components 
of  GRASP  described  in  this  chapter  could  be  extended  for  solving  instances  of 
general  M-adic  assignment  problems  (see  1123]). 


5.3  Computational  Results 

In  this  section,  we  report  on  a computational  experiment  to  evaluate  the  ac- 
curacy and  efficiency  of  a Fortran  77  implementation  of  the  GRASP  described  in 
§5.2. 

The  experiment  was  done  on  a 150MHz  Silicon  Graphics  (SGI)  Challenge  com- 
puter, whose  hardware  configuration  is  summarized  in  Figure  5.3.  The  code  was 
compiled  on  the  SGI  Fortran  compiler  f77  using  compiler  flags  -02  -Olimit  800 
-static.  The  SGI  Irix  5.3  operating  system  was  in  use  during  the  experiments. 
Though  the  machine  used  in  the  experiments  is  configured  with  12  processors, 
processes  were  limited  to  run  on  at  most  one  processor  at  a time.  CPU  times  in 
seconds  were  computed  by  calling  the  system  routine  etime.  Reported  CPU  times 
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exclude  problem  input  and  output  report  times.  The  portable  Fortran  pseudo 
number  generator  of  Schrage  [176]  was  used. 

The  GRASP  for  BiQAP  has  several  parameters  that  need  to  be  set.  We  fixed 
these  parameters  throughout  all  runs  in  the  experiment.  The  RCL  parameters 
used  were  a = 0.25,  (5  = 0.3,  and  A = 1000.  A maximum  of  150  GRASP  iterations 
were  allowed  per  run. 

To  test  the  accuracy  of  the  GRASP,  that  is,  how  the  solution  found  by  GRASP 
compares  with  an  optimal  solution,  we  tested  our  code  on  14  test  problems  having 
known  optimal  solutions,  whose  generator  is  described  in  Burkard  et  al.  [41].  In 
addition  to  the  limit  of  150  GRASP  iterations,  the  algorithm  stops  if  an  optimal 
permutation  is  found.  The  instances  varied  in  dimension  from  n = 10  to  n = 36, 
the  largest  having  1,679,616  entries  in  each  of  its  two  matrices.  These  instances 
were  also  used  by  Burkard  and  Qela  [40]  to  test  several  heuristics  for  the  BiQAP. 
The  GRASP  uses  two  random  number  streams.  The  first  is  used  to  select  the 
A initial  assignments,  while  the  second  is  used  to  build  the  restricted  candidate 
lists  during  GRASP  construction  phase.  In  our  experiment,  ten  replications  of 
the  GRASP  runs  were  done  for  each  test  problem,  each  one  using  a different 

f 

seed  to  generate  the  pseudo  random  number  stream  used  to  select  the  initial  A 
assignments.  The  seeds  used  are  270001, 270002, . . . , 270010.  For  determining  the 
RCL,  the  seed  used  is  12444. 

For  all  14  test  problems  considered,  the  GRASP  found  an  optimal  permutation 
on  every  single  replication,  in  as  few  as  one  GRASP  iteration,  and  in  as  many  as 
138  iterations.  Table  5.1  summarizes  the  computational  experiments.  For  each 
test  problem  instance,  the  table  lists  its  name,  dimension,  cost  of  an  optimal 
permutation,  and  the  minimum,  average,  and  maximum  number  of  iterations  and 
CPU  time  to  find  an  optimal  permutation.  Figure  5.4  shows  the  distribution  of 
GRASP  iterations  for  all  test  problem  replications,  along  with  the  average  number 
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Figure  5.4:  Number  of  GRASP  iterations  to  find  optimal  solution  as  a function 
of  the  dimension  of  the  BiQAP.  Dotted  line  is  average  number  of  iterations.  All 
instances  included. 


Figure  5.5;  GRASP  time  (in  seconds)  to  find  optimal  solution  as  a function  of 
the  dimension  of  the  BiQAP.  Dotted  line  is  average  solution  time.  All  instances 
included. 
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Table  5.1:  Statistics  summarizing  GRASP  runs 


computational  effort  to  find  optimal  permutation 
iterations  CPU  seconds 


problem 

n 

opt  value 

min 

avg 

max 

min 

avg 

max 

biQAP_10 

10 

133216 

1 

1.7 

2 

1.3 

2.3 

3.4 

biQAP.12 

12 

120600 

1 

7.2 

18 

5.3 

23.1 

56.9 

biQAP.14 

14 

251356 

1 

2.2 

4 

8.3 

23.9 

41.5 

biQAP_16 

16 

441696 

2 

13.4 

34 

44.4 

289.6 

749.5 

biQAP_18 

18 

811338 

1 

2.8 

5 

38.5 

138.7 

281.2 

biQAP_20 

20 

1487296 

1 

2.0 

5 

74.8 

184.2 

468.1 

biQAP_22 

22 

3118716 

1 

1.5 

2 

133.6 

267.1 

431.5 

biQAPJ24 

24 

1780378 

1 

28.2 

66 

509.7 

8092.2 

19047.7 

biQAP_26 

26 

5305872 

1 

3.1 

9 

471.7 

1664.3 

4717.1 

biQAP_28 

28 

4606532 

1 

4.7 

15 

568.8 

4130.4 

13683.0 

biQAP_30 

30 

4468860 

4 

27.3 

138 

5370.5 

30344.0 

152500.1 

biQAP.32 

32 

8088327 

1 

6.6 

16 

2644.5 

12585.1 

27822.8 

biQAP_34 

34 

9445329 

4 

14.3 

63 

9922.9 

36936.5 

161505.3 

biQAP_36 

36 

11297520 

2 

21.8 

58 

6102.4 

79281.6 

208924.6 

of  iterations  taken.  Figure  5.5  shows  the  distribution  of  CPU  time  together  with 
average  CPU  time. 

Our  results  indicate  that  the  GRASP  compares  well  with  the  heuristics  de- 
scribed and  tested  in  Burkard  and  Qela  [40].  That  paper  investigated  the  per- 
formance of  Heider’s  method  [97],  the  first  and  best  improvement  methods,  three 
versions  of  simulated  annealing,  two  versions  of  tabu  search  and  a combination  of 
tabu  search  with  simulated  annealing.  In  that  paper,  each  heuristic  is  repeatedly 
tested  on  seven  BiQAP  instances  of  dimensions  n = 10,14,16,20,24,28,32.  Ad- 
ditionally, Heider’s  method,  and  the  first  and  best  improvement  heuristics  were 
tested  on  an  instance  of  size  n = 36.  The  third  version  of  simulated  annealing 
(SimAnn3)  proved  to  be  the  best  among  all  heuristics  tested  by  Burkard  and 
Qela.  SimAnn3  solved  almost  all  of  the  instances  of  sizes  n = 14, 16, 20,  while 
the  average  percentage  of  optimally  solved  instances  for  the  remaining  problems 
is  about  46%.  None  of  the  nine  heuristics  tested  by  Burkard  and  Qela  solved  the 
entire  set  of  instances  of  dimension  n > 12  to  optimality. 
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Figure  5.6:  CPU  time  per  GRASP  iteration  as  a function  of  BiQAP  dimension. 
Computed  values  for  n < 36.  Estimated  time  per  iteration  for  n > 36. 

Although  our  GRASP  code  solved  all  ten  replications  of  all  14  test  problems, 
it  often  required  a large  amount  of  CPU  time  to  find  an  optimal  permutation. 
The  longest  run  in  our  experiment  took  over  two  days  of  CPU  time.  This  can, 
however,  be  remedied  with  distributed  parallel  computing.  Suppose  we  have  ten 
processors  to  run  concurrently,  and  allocate  to  each  processor  a copy  of  the  code, 
a copy  of  the  data,  and  a different  random  number  seed  to  generate  the  random 
number  stream  used  to  select  the  initial  A assignment,  stopping  when  the  first 
processor  finds  the  optimal  permutation.  The  running  time  of  the  10-processor 
system  would  correspond  to  the  minimum  CPU  time  in  Table  5.1.  Those  range 
from  1.3  seconds  for  the  n = 10  problem,  to  a little  under  three  hours  of  CPU 
time  for  the  n = 34  instance.  The  largest  (n  = 36)  test  problem  would  be  solved 
in  less  than  two  hours.  Note  that  because  each  GRASP  iteration  independently 
produces  a solution,  the  heuristic  is  inherently  parallel,  with  a linear  speedup  as 
demonstrated  in  Pardalos  et  al.  [153]. 

On  the  14  instances  considered  in  this  experiment,  10  were  solved  in  a minimum 
of  one  GRASP  iteration,  two  in  a minimum  of  two  iterations,  and  two  in  a minimum 
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Table  5.2:  Optimal  permutations  found  by  all  replications  of  GRASP 


n 


optimal  permutation 


10 

12 

14 

16 

18 

20 

22 

24 

26 

28 

30 

32 

34 

36 


of  four  iterations.  It  is  easy  to  see  that  no  matter  how  many  processors  are  used 
in  a distributed  parallel  implementation  of  GRASP  of  the  kind  described  above, 
the  GRASP  running  time  is  bounded  below  by  the  running  time  of  one  GRASP 
iteration.  Figure  5.6  shows,  for  the  14  test  problems  used  in  the  experiment, 
average  CPU  time  per  GRASP  iteration,  as  a function  of  BiQAP  size,  as  well  as 
estimated  running  times  for  larger  instances.  The  figure  suggests  that  one  GRASP 
iteration  on  an  instance  of  dimension  n = 90  would  take  over  10  CPU  days. 

Finally,  it  is  interesting  to  note  that  for  each  instance,  all  GRASP  replica- 
tions produced  identical  optimal  permutations,  suggesting  that  these  solutions  are 
unique.  Table  5.2  lists  the  optimal  permutations  found. 


In  this  chapter  we  describe  a GRASP  for  finding  approximate  solutions  of  the 
biquadratic  assignment  problem  and  test  a FORTRAN  77  implementation  of  the 
algorithm  on  a set  of  test  problems  with  known  optimal  solutions.  Computational 
experience  with  the  code  indicates  that  the  GRASP  is  effective  in  finding  the 
optimal  permutation  for  these  test  problems.  On  all  test  problems  considered,  the 


5.4  Concluding  Remarks 
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optimal  solution  was  found  on  all  ten  runs,  each  starting  with  a different  seed  for 
the  pseudo  random  number  generator. 

Solution  times  can  be  long  on  the  larger  problems,  with  a single  GRASP  it- 
eration requiring,  for  an  instance  of  dimension  n = 36,  as  much  as  one  hour  of 
CPU  time  on  a SGI  Challenge  computer.  To  mitigate  this  problem,  we  suggest 
three  directions.  Firstly,  a version  of  the  code  that  takes  advantage  of  sparsity 
in  the  problem  matrices  can  be  developed,  as  was  done  for  the  GRASP  for  QAP 
[154].  This  could  make  it  easier  to  solve  large  sparse  problems  that  occur  in  prac- 
tice. Secondly,  as  we  discussed  in  §5.3,  a distributed  parallel  GRASP  could  be 
implemented,  as  was  done  in  Pardalos  et  al.  [153]  for  the  QAP.  Finally,  it  may  be 
possible  to  parallelize  parts  of  the  computation  in  each  GRASP  iteration. 


CHAPTER  6 

MULTIDIMENSIONAL  ASSIGNMENT  PROBLEMS 

6.1  Introduction 

In  this  chapter  we  focus  our  studies  on  finding  good  solutions  for  the  data  asso- 
ciation subproblem.  At  each  discrete  time  interval,  the  data  set  will  be  formulated 
as  a multidimensional  assignment  problem  (MAP)  with  a maximum  likelihood  cost 
function.  A near-optimal  solution  to  each  MAP  will  be  obtained  with  a greedy 
randomized  adaptive  search  procedure  (GRASP)  heuristic.  The  GRASP  has  been 
shown  to  produce  very  good  solutions  quite  efficiently  for  the  quadratic  assignment 
problem  (QAP)  [154],  [129],  [130]  and  the  biquadratic  assignment  problem  [135]. 
Since  the  algorithm  must  run  on-line,  solutions  must  be  found  very  quickly.  As  will 
be  demonstrated,  GRASP  is  easily  parallelized  to  decrease  run-time  substantially. 
The  chapter  is  organized  as  follows.  In  sections  6.2  and  6.3,  we  show  in  detail  how 
the  MTMST  is  modeled  as  an  MAP.  Section  6.4  describes  a GRASP  implemen- 
tation for  solving  general  MAPs  and  the  different  data  structures  developed  for 
efficiency.  Section  6.5  deals  with  the  parallelization  of  GRASP  as  well  as  the  par- 
allel environment  used,  and  section  6.6  contains  some  preliminary  computational 
results. 


6.2  Data  Association  Problem 

Data  association  is  a very  difficult  combinatorial  problem  often  encountered  in 
sensor  fusion  problems,  one  example  being  the  multiTarget  multiSensor  tracking 
(MTMST)  problem.  When  tracking  a single  target  with  a single  sensor,  noisy  mea- 
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surements  are  made  at  discrete,  periodic  points  in  time  from  the  sensor  regarding 
the  moving  target.  The  goal  is  to  establish  a time-continuous  estimate  of  the  tar- 
get’s trajectory.  For  the  MTMST  problem,  noisy  measurements  are  made  from 
an  arbitrary  number  of  spatially  diverse  sensors  regarding  an  arbitrary  number  of 
targets  with  the  goal  of  estimating  the  trajectories  of  all  of  the  targets  present. 
Furthermore,  the  number  of  targets  may  change  (e.g.,  by  wandering  out  of  and 
into  detection  range)  during  the  problem  time  interval  and  the  sensors  themselves 
may  be  in  motion.  For  example  in  Figure  6.1,  two  sensors  observe  three  targets, 
thereby  we  have  three  measurements  from  each  sensor.  The  MTMST  problem  is  to 
partition  all  six  measurements  from  the  sensors  into  three  subsets  that  correspond 
to  targets. 

sensor  2 


Target  state  estimates  for  single-target  single-sensor  tracking  are  obtained  sat- 
isfactorily by  applying  one  of  several  variants  of  the  Kalman  filter.  However,  no 
method  known  can  consistently  obtain  good  trajectory  estimates  for  the  more 
general  MTMST  problem. 

The  MTMST  has  only  recently  been  addressed  [61],  [62].  In  these  and  other 
papers  the  MTMST  problem  has  generally  been  divided  into  2 subproblems:  state 
estimation/propagation  and  data  association.  The  combinatorial  nature  of  the 
MTMST  problem  results  from  the  data  association  subproblem;  that  is,  given  that 
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each  sensor  m,  m = 1 . . . M,  makes  measurements,  rii  7^  ^ j,  how  do  we 

correctly  partition  the  measurements  into  all  of  the  targets,  ensuring  that  all  of  the 
measurements  are  used  exactly  once?  Total  enumeration  of  all  possible  partitions 
is  obviously  unrealistic. 

In  MTMST  a data  association  problem  is  presented  at  discrete,  periodic  mo- 
ments in  time.  We  expect  that  by  obtaining  the  correct  partition  or  association, 
the  data  from  all  of  the  sensors  may  be  combined  in  some  manner  to  make  esti- 
mates of  target  positions  that  are  more  accurate  than  that  obtained  from  a single 
measurement. 

In  this  section  we  briefly  develop  the  Data  Association  problem  formulation 
for  a MTMST  problem.  The  development  is  given  in  detail  in  the  paper  by  Mur- 
phey,  Pardalos  and  Pitsoulis  [144].  Other  treatments  can  be  found  at  Pattipati  et 
al.  [158],  and  Poore  et  al.  [161,  162]. 

At  some  time  tk,  each  sensor  makes  noisy  measurements  of  a set  of  objects 
that  are  either  true  targets  or  spurious  measurements  (false  alarms).  These  mea- 
surements are  used  to  estimate  the  true  kinematic  state  (position)  of  targets.  The 
data  association  process  partitions  object  observations  into  targets  and  false  alarms 
such  that  a single  list  of  true  targets  results.  The  technique  discussed  here  seeks 
to  maximize  the  likelihood  of  a partition  of  measurements  into  targets  and  false 
alarms.  Once  the  measurements  are  associated  with  targets  and  false  alarms,  es- 
timates of  the  true  target  positions  may  be  made  by  combining  the  data  from  all 
of  the  sensors  associated  to  each  target  in  some  weighted  fashion  (however,  in  this 
chapter  we  treat  the  partition  problem  only). 

We  define  the  following  terms  for  the  MTMST  problem; 

• m:  the  sensor  index 


• j:  the  target  index 
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• M:  the  total  number  of  sensors 

• J:  the  true  total  number  of  targets 

• Apriori  ^ J'-  f^e  a priori  estimate  of  the  number  of  targets 

• Xj,  j = 1 . . . J\  the  true  state  of  the  target 

• Xj,  j = 1, . . . , Japriori  • the  estimated  state  of  the  j**  target 

Q 

• Zm,im  ^ ) *m  = 1 ■ ■ ■ i^m'-  ^ Set  of  Cartesian  measurements  from  sensor 

m 

• Vm,irn'-  the  measurement  noise  of  sensor  m with  respect  to  observation  im 
distributed  normal  with  zero  mean 

• Rx  : covariance  matrix  of  Vm  % 

• Rsm-  error  covariance  matrix  of  all  sensor  m measurements  in  spherical 
coordinates  (measurements  are  made  in  spherical  coordinates  and  Rx^  is 
obtained  for  Cartesian  coordinates  by  at  transformation  of  Rsm  as  discussed 
in  [144]) 

• Wm’  false  alarm  random  variable 

• Pfa^'  probability  of  false  alarm,  p{wm  > 1)  = Pfa^,  assumed  constant  and 
known 

• Pom-  probability  of  sensor  m detecting  any  target,  assumed  constant  and 
known 

We  will  assume  that  Vm,im  Wm  are  mutually  independent  and  each  is  inde- 
pendent from  one  sensor  to  the  next.  If  a sensor  fails  to  detect  a target  it  is  said 
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to  have  miss  detected.  We  wish  to  reserve  a place  in  our  measurement  array  to 
represent  this  situation.  Define 


^m,0  — ^ 


1,  if  sensor  m does  not  detect  a target, 
0,  else. 


Clearly,  P{zmfi  = 1)  = 1 — Pom-  Define  a Boolean  variable  to  represent  missed 
detections; 


^m,im  ~ ^ 


1,  if  index  im  — 0 (i.e.,  ^ missed  detection), 

0,  otherwise. 


(6.1) 


Let  o:m,im  be  a decision  variable  such  that: 


^Tn,im  — ^ 


1,  when  measurement  im  of  sensor  m is  a false  alarm, 
0,  when  measurement  im  is  a true  target. 


(6.2) 


Each  measurement  is  defined  as 


^m,im  ~ ^Tn,im)[^j  d"  d" 


(6.3) 


Because  the  measurement  process  is  noisy,  we  cannot  know  xj  precisely.  Hence,  we 
assess  the  likelihood  that  a given  measurement  is  associated  with  a given  estimated 
target  which  we  denote  by  However,  as  seen  in  equation  (6.3)  the  noise 

is  additive  and  we  know  that  Vm,im  is  Gaussian.  So,  given  that  the  measurement  is 
not  a false  alarm  or  missed  detection,  Zm,iml^j  is  distributed  Gaussian  with  mean 

Xf 


1 


1 

2 


p{^m,im  I ^j)  ~ 


(6.4) 
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6.2.1  Multidimensional  Assignment  Formulation 


The  set  of  all  measurements  from  the  sensor  is 


' • • ^m,nm  } 


The  set  of  measurements  from  all  sensors  is  the  array 


Z — |Zi,  ^2, . . . , Zmi  • • • , Zm } 


Define  an  M-tuple  of  measurements  in  Z to  be 


in  other  words,  exactly  one  measurement  from  each  sensor.  Observe  that  by  defin- 
ing the  missed  detection  Zm,o  we  are  always  able  to  form  M-tuples  in  Z,  no  matter 
how  many  sensors  actually  see  a given  target,  since  the  balance  of  sensors  can  be 
said  to  have  miss-detected. 

Define  a Boolean  decision  variable  that  establishes  whether  or  not  all  of  the 
elements  in  a given  M-tuple  are  noisy  measurements  of  the  same  object,  that  is, 
if  a tuple  is  target  homogeneous. 


Pil,i2 ^ 


1,  if  the  tuple  is  target  homogeneous. 


0, 


otherwise. 


(6.5) 


The  likelihood  that  a given  M-tuple  is  target  homogeneous  is: 


M 


m=l 


(6.6) 
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Without  loss  of  generality,  assume  that  each  target  is  always  detected  by  at  least 
one  sensor.  Then  A(po,o,...,o  = = 0- 

Consider  a partition  of  Z into  measurements  corresponding  to  true  targets  and 
those  corresponding  to  false  alarms.  Denote  the  set  of  all  false  alarm  measurements 
as  Zf.  A feasible  partition  of  Z will  be  denoted  as: 

T ^ ^ 9,1,..., Tfi  1,2,...,  A/ Z j \ . (6.T) 

As  an  example,  consider  the  case  in  Figure  6.2  of  three  sensors  observing  two  true 
targets.  One  possible  partition  of  the  measurements  assumes  Japriori  = 2 targets; 
the  first  target  detected  by  all  3 sensors  (measurements  2:1,1, -Z2, 2, -2^3,1),  the  second 
detected  by  only  the  first  2 sensors  (21,3,2:2,1,2:3,0),  and  a false  alarm  by  sensor 
1 (21,2)-  Another  partition  implies  Japriori  = 1 target  which  is  detected  by  all  3 
sensors  (measurements  21,1,  ^2,2, 23,1),  2 false  alarms  from  sensor  1 (21,1,  Zi, 3)  and 
1 false  alarm  by  sensor  2 (22,1).  Still  other  partitions  for  many  values  of  Japriori 
exist. 

^2.2  ■ 

^2.1  ■ 

^1.2  © 


Figure  6.2:  Example:  3 sensors  and  2 Targets 

The  feasibility  of  a partition  requires  that  each  measurement  belongs  to  either 
a target  tuple  or  a false  alarm  but  not  both,  and  that  each  M-tuple  corresponds 
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to  one  and  only  one  target.  These  two  conditions  are  equivalent  to: 


■■■  Pii,i2,-,iM  1 + — 1)  (6-8) 


im  = . . . ,nm  and  m = 1,2, . . . , M. 


In  essence,  for  any  measurement  im  from  any  sensor  m there  must  either  exist  an 
M-tuple  for  a valid  target  that  contains  this  measurement  or  else  the  measurement 

is  a false  alarm.  There  are  ni  + U2  H h um  such  feasibility  constraints  each  with 

(niri2  . . . rim-iUm+x  ■ ■ ■ um)  + 1 terms. 

We  wish  to  find  the  most  likely  partition  of  Z,  hypothesizing  that  if  Tapnon  > J , 
then  the  resulting  tuples  will  correspond  to  our  list  of  true  targets.  Consider  the 
likelihood  of  a partition  7 to  be  the  joint  probability  of  all  target  tuples  and  all 
spurious  measurements  in  the  partition.  Since  targets  are  independent  of  one 
another  as  are  false  alarms,  the  joint  probability  is  simply  the  product  of  the 
individual  tuple  and  false  alarm  densities: 


the  joint  probability  that  all  measurements  are  false  alarms  (i.e.  there  are  no 
targets).  Denote  this  by: 


L{i)  = n 

_ic% i TLf  GO' 


Zl,l2  5 


iori  (7)  being  the  assumed  number  of  targets  in  7.  Furthermore,  consider 


M 


i(7o)  = n (Pi-.)"'  ■ 


(6.10) 
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If  we  denote  the  set  of  all  feasible  partitions  as 


^ {Ti  ) T2)  • • • T'I’m  } > 


(6.11) 


where  is  the  total  number  of  partitions  (a  very  large  number),  then  finding  the 
maximum  likelihood  normalized  to  the  no-target  case  is  simply: 

(6. 

which  is  the  same  as: 


max 

Tsr 


Hi) 

i(7o) 


min  (lnL(7o)  — lnL(7)}. 


(6.13) 


From  equation  (6.9)  we  compute 


lnZ/(7)  'y  ^ ln(A(pjj^j2,...,iM> ^i)) 

^*1 ,12 

M 

~ ^ ^ (^m  “ ^apriori  ) 

m=l 


(6.14) 


where  the  first  term,  which  we  call  the  unnormalized  negative  log  tuple  cost,  is: 


M 


lnA(pij^j2 ~ y 


m=l 


(6.15) 


1^(1  ^Dm) 


We  call  the  term  MC{zm,im,xj)  the  negative  log  measurement  cost  and  it  is: 


MC{Zm,im,x 


(6.16) 
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Also  from  (6.10)  we  obtain 


M 

lnL(7o)  = In  (P/a„).  (6.17) 

771=1 

Substituting  (6.15)  into  (6.14)  then  (6.14)  and  (6.17)  into  (6.13)  we  obtain  the 
normalized  negative  log  cost  function: 

(6-18) 

where 


M 

Qi,Z2v,*M  “ ^ “ ^m,im)^^i^m,im,Xj)  + ^m,im  ll^(l  ~ ^Dm) 

m=l  (6.19) 

4"  '^apriori  ) • 

The  is  effectively  the  cost  of  tuple  Since  defines 

whether  or  not  a tuple  corresponds  to  a target,  it  is  essentially  a Boolean  decision 
variable  with  corresponding  cost  The  objective  function  in  (6.20)  thus 

follows.  The  constraints  in  (6.21)  come  directly  from  feasibility  conditions  previ- 
ously defined  in  (6.8).  The  result  is  a multidimensional  assignment  problem  (MAP) 
that  chooses  tuples  of  measurements  which  minimize  the  negative  log  likelihood 
cost  function: 


mm 


E 


-'ll  ,i2,...,i7v/ 
771  772 


Ql  ,72  v>*M  Al  >72,. ..,7m 


77m  — 1 77m-f-l 


77m 


S XI  ■ ■ ■ X X ■ ■ ■ X Piui2,...,iM 

ij— 0 I2— 0 im— 1“0  ^m+1— 0 0 


Pil  ,i2).--)*A/  ^ ^ f J • • • ? ’^TTll  f ? • • • J A4". 


(6.20) 

(6.21) 

(6.22) 
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6.3  Computing  the  Costs 


Obtaining  the  costs  in  (6.19)  requires  that  we  have  an  a priori  estimate  of  the 
target  position  xj.  Given  an  M-  tuple  of  measurements  the  linear  least 

squares  estimate  of  the  target  position  Xj  is: 


iu 


mm 

X 


9 ^m,im 


- \n{2TT\Rl^iJ) 


(6.23) 


+ ^ [ln(l-PD.) 


im  —0 


where  we  say  that  if  = 0 then  sensor  m has  miss  detected  hence  m is  not 
included  in  the  first  summation  but  included  in  the  second  summation.  Since  x 
is  unknown  in  a maximum  likelihood  function,  and  the  measurement  z is  a linear 
function  of  the  target  state  x,  finding  xj  that  minimizes  the  function  on  a;  is  a 
linear  least  squares  problem. 

To  obtain  xj,  we  simply  set  the  gradient  with  respect  to  x of  the  function  in 
equation  (6.23)  to  zero  and  solve  for  x.  Since  the  natural  log  terms  are  not  a 
function  of  x,  they  are  disregarded. 


771=1,. ..,M: 
im 


2 


m 


= 0, 


This  yields: 


771=1,...,M: 

im 


777=1,. ..,M: 
im  ^0 


X 
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Define  the  following: 


E « 


-1 


im  7^0 


(6.24) 


Solving  for  x = Xj  yields  the  least  squares  estimate  of  x to  be: 


Xi  = 72. 


-1 


m 


(6.25) 


Now  Xj  may  be  used  to  compute  the  cost  of  tuple  equation  (6.19). 

However,  we  still  do  not  know  what  Japriori  is.  Observe  that  no  matter  what 
tuple  is  selected,  the  last  term  in  equation  (6.19)  will  be  constant  for  fixed  value 
of  -^apriori  • Furthermore,  by  varying  Japriori  » the  costs  will  vary  also.  To  find  the 
value  of  Japriori  fhat  minimizes  the  cost  function  in  concert  with  the  partition,  we 
could  assume  several  values  of  Japriori  > rninimize  the  resulting  MAPs,  and  then 
pick  the  one  with  the  lowest  cost  solution.  We  would  only  need  to  check  the  values 
minm{nm}  < Japriori  < and  these  tasks  could  be  done  in  parallel. 

Since  the  focus  of  this  chapter  is  on  solving  the  MAP,  we  will  henceforth  assume 
a value  of  Japriori  = J and  find  a solution  for  the  resulting  MAP. 

Finally  a word  on  the  size  of  the  cost  vector.  If  we  assume  that  there  are  no 
false  alarms,  the  number  of  costs  we  will  need  to  compute  will  be 


(rij  + 1), 

(where  1 is  added  to  account  for  missed  detections).  In  the  worst  case  (no  missed 
detections),  this  will  mean  that  (J  + 1)^  costs  will  be  computed  where  J = rij  Vi 
is  the  number  of  targets  and  M is  the  number  of  sensors.  Thus  the  length  of  the 
cost  vector  will  increase  much  faster  as  a function  of  M than  as  a function  of  J. 
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Eventually  for  large  enough  M,  computing  the  cost  vector  becomes  a difficult  task. 
Preconditioning  can  be  accomplished  (as  discussed  in  [144])  by  using  “gates”  to 
exclude  most  tuples  from  consideration,  thereby  yielding  a sparse  cost  vector.  Since 
the  focus  of  this  chapter  is  on  solving  the  MAP,  we  will  not  discuss  preconditioning 
techniques  further. 


6.4  The  GRASP  Implementation 

GRASP  has  been  implemented  to  solve  quadratic  assignment  problems  (QAP)s 
with  dense  [130,  129]  and  sparse  coefficient  matrices  [154],  while  an  implementa- 
tion for  solving  the  biquadratic  assignment  problem  can  be  found  in  Mavridou  et 
al.  [135].  A parallel  implementation  of  a GRASP  for  solving  the  QAP  is  described 
in  Pardalos  et  al.  [153]. 

In  what  follows  the  optimization  problem  formulation  of  the  M-dimensional 
assignment  problem  will  be  presented,  first  as  an  integer  program  (IP)  and  then 
using  permutations.  Let  ni,  n2, . . . , um  be  integers  which  define  the  ranges  in  each 
dimension.  The  input  data  consists  of  one  cost  array  C G where 

without  loss  of  generality  assume  that  n\  < U2  < • ■ • < um-  The  IP  formulation 
of  the  M-dimensional  assignment  problem  is  the  following: 


m 

min  2^  ■ ■ 

riM 

C ■ X-  ■ 

(6.26) 

il=l 

2M  = 1 

712 

flN 

s.t.  ^2 ' ■ 

E- 

i\ '"iM 

1,  ii 

1 , . . . , Tl\ , 

(6.27) 

12  = 1 

ZM  = 1 

Til 

rik-i 

flk  + l 

riM 

E- 

■ E 

E •• 

E- 

= 1 

(6.28) 

h=l 

-1-1  = 1 

zm=1 

for 

^/c  — Ij 

• * • 5 

and  k - 

= 2,...  ,M-  1 

(6.29) 

Til 

flM-\ 

E- 

• E 

"'IM 

= 1, 

= 1)  • • • ) 

(6.30) 

il=l 

L 

T • 

r e {0, 

1}  for  all  i\  • ■ 

■ iM- 

(6.31) 
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Let  be  the  set  of  all  arrays  X = {xi,...i^)n,x-xnM  such  that  their  elements 
satisfy  the  constraints  (6.27)  through  (6.31).  The  above  IP  formulation  of  the 
MAP  however  is  not  very  convenient  with  respect  to  algorithmic  issues;  it  does 


however  provide  an  insight  to  the  nature  of  the  problem.  Define  the  sets  of  integers 
Ni  :=  {1, 2, . . . , Hi},  i = 1, 2, . . . , M,  and  let  11;^,  be  the  set  of  all  ni-permutations 
Pi  : iVi  Ni.  Note  that  |riArJ  = , for  each  i = 1, . . . , M.  Given  pi, . . . ,pm 

let  P = (pi, . . . ,Pm),  that  is  P can  be  regarded  as  a rzi  x M matrix  with  columns 
the  ni-permutations  Pi, . . . ,PM) 


^Pi(l)  P2(l)  •••  Pm(1)  ^ 

Pi(2)  P2(2)  •••  Pm(2) 


(6.32) 


\Pi{ni)  P2(ui)  •••  PM{ni)J 


Now  given  rii  < ri2  < ■ ■ ■ < um,  and  denoting  by  = (1,2,...,  rii)  the  identity 
permutation  of  Ni,  let  be  the  following  set 


:=  {P  = (pi . . . ,Pm)  I Pi  = eNi,Pi  e UNi,i  = 2, . . . ,M}. 


^ ”4-v7.  Note  that  there  is  a one-to-one  corre- 


It  can  be  seen  that  P^  = rii  o t m 

1 U=2  (rii-ni)\ 

spondence  between  the  elements  of  and  P^  for  given  rii  < ri2  < 


since  given  any  P G P^  we  can  construct  an  X G X^  as 


M 


Xii'.'ij^  ^ 


1 if pj{ii)  =ijj  = 2,...  ,M, 
0 otherwise. 


(6.33) 


for  = 1,...  ,ni,  and  vice  versa.  We  can  therefore  formulate  the  MAP  using 


permutations  as  follows: 
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(6.34) 


S.t.  Pi  G z = 2, . . . , M. 


A solution  to  the  MAP  using  the  above  formulation  will  be  any  P G such  that 


the  above  objective  function  is  minimized. 

6.4.1  Storing  the  C Array 

It  immediately  follows  from  the  nature  of  the  problem  that  an  efficient  way 
of  handling  the  large  data  is  required.  There  are  two  basic  problems  in  handling 
the  input  data  of  an  instance  of  the  MAP.  First,  multidimensional  arrays  are  not 
desirable  in  terms  of  memory  requirements,  and  second,  the  variable  dimension  M 
poses  implementation  problems.  Both  of  these  problems  are  solved  if  we  transform 
the  multidimensional  array  into  a one-dimensional  array.  Let  tt;  := 


/ = !,...  ,M.  The  array  C G ]R"i^"2x  - xnM  jg  stored  as  an  one-dimensional  array 
c G in  column  format.  For  example  if  C G then  the  array  c G IR®  will 

be 


In  addition  to  the  sets  of  integers  N{,  z = 1, . . . , M defined  previously  define  also 
the  set  N :=  {1,2,...  , tt}.  The  set  NiX-  • -xN^  can  be  regarded  as  the  index  set  of 
C while  N the  index  set  of  c.  The  transformation  between  an  index  (zi,  Z2, . . . , zm) 


of  C to  an  index  A:  of  c is  given  by  the  following  function  f : Ni  x • • ■ x Nm  N 


M 


(6.35) 
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The  inverse  (from  an  index  A:  of  c to  an  index  (?i,  Z2, . . . , im)  of  C)  can  be  obtained 
in  a similar  fashion  from  the  function  : N ^ Ni  x ■ ■ ■ x Nm,  the  inverse  of  /, 
which  is  defined  by 


^ ~ 1 ~ ^ T^M-liiM-l+l  — 1) 


+ 1 


\ 


f \ 


r\k)  := 


ttm-1 


+ 1 


'771 


V 


A;— 1 
TTM-1 


+ 1 


/ 


\ iM  / 


Remark  6.1  The  function  f is  a bijection. 

Proof  6.1  Since  |A^i  x • • • x Nm\  = |iV|  = 'k  by  construction,  for  f to  be  bijective 
it  suffices  to  show  that  for  any  two  {%i, . . . ,1m),  {ji,  ■ ■ ■ ,Jm)  ^ -^i  x • • • x Nm, 
{ii, . . . , 7^  (ii)---  (Jm)  (i-^-  distinct  elements),  we  have  f{ii,...  ,iM)  7^ 

fih,--  - Jm)-  Clearly  this  is  true  for  M = 1,  since  f{ii)  = ii  ^ ji  = f{ji). 
Suppose  that  is  true  for  M and  let  (ii, ...  + 1)  7^  {ji,  ■ ■ ■ ,jM  + 1)-  Then  we 

have 


f{ii, ...  , zm  + 1)  — /(*!,  • • • , Im)  + — 1)> 


5 * • • 1 jAf  1)  y* (jl  5 * * * ) Jm)  ”b  M (JM+1  1) • 

There  are  three  cases:  i)  (ii,...  ,iu)  7^  (ji)---  iJm)  o,nd  iu+i  = 3m+\,  H) 
(ii, ...  ,1m)  = (ii,  • ■ ■ , Jm)  and  therefore  Im+i  # Jm+i,  and 

Hi)  (ii, . . . , zm)  7^  (ji,  • • • , Jm)  and  Zm+i  7^  Jm+i-  In  all  of  the  cases  /(*i, . . . Jm  + 
1)  7^  /(ji)---  i3m  + 1)  since  we  have  respectively  for  the  first  case  the  induc- 
tion hypothesis  and  for  the  second  case  iM+i  7^  Jm+i-  For  the  third  case,  note 
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that  /(ii,  • • • ) / (ji)  • • • ) Jm)  ^ [1)  • • • ) '^m]  (if^d  ~ 1)>  '^M{jM+i  ~ 1)  ^ 

[0,  ttm,  . . . , (um+i  — 1)7Ta^],  which  imply  the  following  inequalities  considering  that 
we  have  distinct  elements 


^ — I ./(^l ) • • • ) ^m)  f i,j\i  • • • t JM  ) I ^ 1 ) 


7Tm(*M+1  ~ 1)  — T^mUm+I  — 1)1  > T^M- 


Therefore,  we  have  that 


'^AI  (^A^+1  ^ ^ AI  ( JA/+1  ^ (y* (^1 ) ■ • ■ ) J A[^  (^1 ) * * * ) IaT ) ) j 


which  implies  that  f{ii, ...  , + 1)  7^  /(jii  • • • + !)• 

From  / and  it  is  clear  that  we  can  construct  the  c array  as  c{i)  = 
C{f~^{i)),i  = 1,...  ,7T. 

6.4.2  The  Index  Structure  of  c 

Assume  that  we  are  given  an  element  (^1,^2)  • • • , I At)  of  C,  and  for  a given  k we 
wish  to  generate  the  set 


N D Si^  :=  {i  e N \ f ^{i)  = {ii, ...  ,ik, , iju)}, 


that  is  the  set  of  all  elements  in  N such  that  their  corresponding  element  in  Ni  x 
• • • X Nm  contains  ik.  A direct  approach  would  be  to  generate  the  set  C 
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Ni  X ■■■  X Nm  defined  as 

{ (^1 ) • • • > ) • • • ) ) I f j 1 , . . . , Tlj , j 1 , . . . , M , j ^ /u } , 

and  then  take  the  image  of  under  /,  Si^.  = f{Sl  ),  a process  which  will  require 
0[M^)  time.  The  way  that  c is  constructed,  as  described  in  §6.4.1,  it  follows  that 

rC 

its  index  structure,  that  is  the  relationship  between  the  indices  of  c and  those  of 
C,  is  well  defined.  Let  us  define  for  expository  purposes,  an  ordered  7r-tuple  which 
corresponds  to  the  index  set  77  of  c,  as  7 :=  (1, 2, . . . , tt  — 1,  tt).  For  every  z G 7 
there  is  a corresponding  element  • • • i*m)  in  the  index  set  of  C.  Define  the 

following  for  every  dimension  k = 1,2,...  , M and  element  ik  € Nk'- 

l)^A:— 1 T 1) 


l^k  ■ 


7T 


According  to  the  above  values,  for  any  ik  we  can  identify  the  following  Tr/^-i-sub- 
tuples  contained  in  7, 


7 = (1,2, . . . ,5J^. . ;Bll,.. . ,7T  - 1,7t), 


where  B}'‘  = {{I  - l)7rjt  + <7j^,  {I  - l)7rfc  + + 1, . . . , (/  - l)7Tfc  + <7*^  + nk-i  - 1)  for 

I = 1, . . . ,Uk.  So  for  every  ik,  I contains  Uk  sub-tuples  of  size  starting  from 
the  element  and  occurring  every  tt*,  elements. 

Remark  6.2  S’jj,  = Uti 

Proof  6.2  Since  Si^  = it  suffices  to  show  that  = Uti where, 

considering  the  fact  that  f is  a bijection,  it  would  be  true  if  for  any  {i\, . . . ,iu)  ^ 
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we  have  / defined  as  in  (6.35)  and  any 

{ii, . . . ,ik,  ■ ■ ■ , iu)  £ si  we  have 


M 

3=2 

M 

f (^1 ) * * * 5 ik—1^  ^/c— 1 (^A:  ^ ^ '^j—1  {ij  1) • 

j=A:+l 

The  ranges  for  f{ii,...  ,ik-i)  and  Z)^fc+i ~ 1)?  are  [1,2,...  ,7Tfc_i]  and 
[0,  TTfe,  27Tfc, . . . , TTfc+i  — TTfc,  TTfc+i, ...  , 7T  — TTfc]  respectively,  while  the  term  7Tk-i{ik  — 
1)  is  constant.  Combining  these  ranges  we  can  see  that  f{ii,...  ,ik,-..  ^m)  ^ 

[B\\  Bt  ■ ■ ■ , Bil]. 

It  follows  from  remark  6.2  that  we  can  generate  Ur=i  therefore  in 

0{—)  time. 


6.4.3  GRASP  Construction  Phase 

The  objective  function  of  the  MAP  is  nothing  else  but  the  summation  of  ni 
elements  of  C such  that  the  permutation  constraints  are  satisfied.  In  the  construc- 
tion phase  we  will  construct  a solution  in  a greedy  randomized  manner.  Let  us 
define  the  set  of  already  made  assignments  upon  making  the  assignment  as 

Fk:=  {ie  N \ f~^(i)  = {pi{j), . . . ,PM{j))  for  some  j = 1, . . . , A:  - 1}, 
and  the  set  of  infeasible  assignments  resulting  from  assigning  s in  the  solution 

M 

As  :=  {i  e N \ for /“^(s)  = (n,...  ,iM)fi  G IJ-S'iJ. 
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Our  RCL  will  be  represented  by  a combination  of  a doubly  linked  list,  with  a 
pointer  array.  The  array  keyO  is  the  c array  sorted  in  increasing  order,  and  the 
invO  array  is  just  a pointer  array  that  shows  the  position  in  the  keyO  of  any 
index  of  c.  The  arrays  prevO  and  nextO  are  used  for  the  doubly  linked  list,  that 
is  to  move  in  the  keyO  array. 


c() 

inv() 


prevO 

key() 

next() 


10 

4 

21 

2 

13 

. 

c(n-3) 

c(n-2) 

c(n-1)  c(n) 

3 

2 

5 

1 

4 

. 

; 

• 

• * I • ! 

i : : : : 

YT  lYT  lYT  YIT  VI  Y 1 Y lY  1 Y 1 

/ 
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2 
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4 

. 

n-4 

n-3 

n-2 

n-1 

4 

2 

1 

5 

3 

• « • 

• 

• 
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3 

4 

5 

6 

• • • 

n-2 

n-1 

n 

/ 

1 A 1 A 1 Al Al a.  A 1 A 1 A 1 A 

Figure  6.3:  Doubly  linked  lists 


Example  6.1  Consider  for  example  Figure  6.3  where  the  c array  and  the  inv() , 
prevO  , keyO  and  nextO  arrays  are  as  shown.  Assume  that  we  make  the  first 
assignment,  and  choose  randomly  among  the  first  [aTrJ  smallest  elements  o/keyO, 
and  assume  that  we  pick  the  element  key(2)=S.  Then  Fi  = {2}  and  assume  that 
^2  = {2, 5},  where  A2  is  generated  as  described  previously.  We  update  our  RCL 
as  follows: 

For  element  2, 
key(inv(2))=-l, 

prev(next (inv(2) ) )=prev(inv(2) ) , 
next (prev(inv(2) ) )=next (inv(2) ) , 
and  for  element  5, 
key(inv(5) )=-l , 

prev(next (inv(5) ) )=prev(inv(5) ) , 
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procedure  ConstructionPhaseCa) 

1 T = 7T  : To  = 0; 

2 do  /c  = 1, . . . , ^ 

3 t =random[l,  [oTj]; 

4 s=startkey; 

5 do  i — ...  it  — ^ 

6 s =next(s)  ; 

7 od; 

8 s =key  (s) ; 

9 = r^_i  U {s} ; 

10  GenerateDelta(s,  Aj,)  ; 

11  for  i ^ As 

12  T = T-1; 

13  UpdateLists(z,prev,next,key) ; 

14  A,  = A,/{i}; 

15  rof; 

16  od; 

17  return 

end  Construct ionPhase; 


Figure  6.4:  Pseudo-code  for  GRASP  construction  phase 
next (prev (inv (5) ) ) =next ( inv (5) ) . 


Every  time  we  ehoose  an  element  from  the  keyO  array  we  proceed  using  the 
prevO,  nextO  arrays. 

The  construction  phase  is  shown  in  pseudo-code  in  Figure  6.4.  In  line  1 we 
initialize  the  size  or  our  RCL,  and  the  set  P.  In  the  loop  defined  by  the  lines  2-16, 
all  the  assignments  are  made.  After  generating  a random  number  in  line  3,  in  lines 
4-8  we  move  in  the  doubly  linked  list  to  retrieve  the  index  s which  corresponds  to 
our  current  assignment.  We  update  F in  line  9,  and  in  line  10  we  generate  the  set  of 
infeasible  assignments  due  to  s,  the  set  A^.  The  procedure  GenerateDelta(s,  A^) 
is  done  as  described  in  6.4.2.  In  lines  11-15  we  update  the  RCL,  by  excluding 
from  the  doubly  linked  list  the  infeasible  elements  contained  in  A^,.  The  procedure 
UpdateLists(i,prev,next ,key)  is  described  earlier. 
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6.4.4  Local  Search  Phase 

After  completion  of  the  construction  phase,  we  have  a feasible  solution  which 
is  consisted  of  M ni-permutations.  In  the  local  search  phase,  we  try  to  improve 
each  solution  constructed  in  phase  1 by  searching  its  neighborhood  for  a better 
solution. 

Let  p be  a r-permutation  of  n elements  (r  < n).  Define  the  difference  between 
p and  a distinct  r-permutation  q,  to  be 

<^r(p,  q)  ■=  {i  I p{i)  q{i),  i = , r}.  (6.36) 

Define  the  distance  between  them  as 


driP^q)  ■=\^r{p,q)\-  (6.37) 

The  k- exchange  neighborhood  of  p is  the  set  of  all  permutations  q such  that 

dr{p,q)  < k, 

Nk{p)  ■=  {q  I dr{p,q)  < k,  1 <k  <r}.  (6.38) 


Note  that  |A2(p)|  = (0  -f  r(n  — r).  For  a given  P G P 
neighborhood  as  the  set 


M 


N2{P)  :=  {Q  € P^  I dM{Pj,qj)  < 2 for  all  j = 1 


define  its  2-exchange 


,M},  (6.39) 


while  |JV2(P)|  = n",  ((7  ) + i^Minj  — nM))-  A local  search  algorithm  based  on 
this  neighborhood  definition  would  run  in  exponential  time,  therefore  we  have  to 
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employ  a different  neighborhood  definition  which  is  smaller  in  size.  Let 


N2{P)  D N2{P)  :=  {Q  e 


dhiiPj,  Qj)  < 2 for  some  j = 1,...  , M}.  (6.40) 


Therefore  each  Q € N2{P)  differs  from  P in  only  one  column.  In  this  case  we  have 
|iV2(P)|  = M(7)+nME,^  j(rzj  — Um)-  For  every  Q E N2{P)  we  can  associate 
a triplet  (j,k,l)  where  j — 1,...  ,M,  k = 1,...  ,um  and  I = k,...  ,rij,  which 
indicates  that  Pj{k)  — qj{l)  and  pj{l)  = qj{k).  HI  <um  then  d,M{Pj,qj)  = 2 while 
if  Um  < I < Uj  then  duiPj,  qj)  — 1- 

Next  we  define  the  gain  (either  positive  or  negative)  between  two  solutions  in 
a neighborhood.  Let 

ni 

^{P)  — y ]^ip2(i)-PM-l(i)PM(i)  (6-41) 

i=l 

for  some  P G P^.  Note  that  Z{P)  is  equivalent  with  the  objective  function  defined 
in  (6.35).  Then  it  is  immediate  that  for  any  Q G N2{P) 

^{P)  ~ Z{Q)  = y ^ ^pi(i)—PM(i)  ~ (6.42) 

ieS(pj,qj) 

and  due  to  the  definition  of  N2{P)  there  is  only  one  j such  that  5{pj,qj)  ^ 0. 
Having  defined  (6.40)  and  (6.42),  a local  search  procedure  for  finding  local  optima 
is  straightforward.  Given  any  P we  generate  the  gains  Z{P)  — Z{Q)  for  every 
Q G N2{P),  and  we  pick  the  Q with  the  largest  positive  gain.  The  procedure  is 
repeated  in  N2{Q)  until  no  positive  gains  can  be  found,  that  is,  our  current  solution 


is  the  locally  optimum. 
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6.5  Parallel  GRASP 

Parallel  programming  emerged  as  an  alternative  way  to  increase  the  computing 
power  of  existing  technology  processors,  by  employing  more  than  one  processor 
for  a given  computation  simultaneously.  Both  software  and  hardware  issues,  like 
balancing  the  work  equally  among  the  processors,  fast  data  communication  and 
memory  sharing,  determine  the  efficiency  of  a given  parallel  program.  A measure 
of  efficiency  that  has  been  adapted  is  the  concept  of  speedup,  where  if  tj  is  the 
computational  time  needed  to  perform  a computation  on  i processors  in  parallel, 
then  the  speedup  for  m processors  working  in  parallel  is  defined  as  A close 
to  linear  speedup  is  thus  desired,  since  that  would  imply  that  the  computational 
effort  is  distributed  perfectly  among  the  available  processors. 

There  are  various  ways  to  employ  parallel  processors  in  parallel,  depending  on 
whether  each  processor  has  its  own  memory  or  if  memory  is  physically  shared, 
if  communication  between  processors  is  through  hardware  or  software  and  other. 
These  issues  determine  the  parallel  computation  model,  such  as  data  parallelism, 
shared  memory  and  message  passing.  In  the  message  passing  computational  model, 
a set  of  processors  each  with  its  own  memory  is  available,  where  some  sort  of  com- 
munication between  them  such  as  a network  is  used.  This  model  can  be  used  in 
existing  networks  of  computers,  and  it  is  highly  portable.  In  parallelizing  GRASP 
for  solving  the  MAP,  we  employ  the  message  passing  model,  since  the  communi- 
cation between  processors  is  minimal. 

The  message  passing  interface  (MPI)  standard,  is  a library  that  contains  rou- 
tines and  macros  that  can  be  used  in  C and  Fortran  77,  that  allows  a number  of 
processors  that  are  connected,  in  a network  to  be  used  in  parallel.  MPI  includes 
functions  that  allow  point-to-point  communications  involving  pairs  of  processors 
and  collective  communications  which  involve  groups  of  processors.  A detailed  de- 
scription of  the  MPI  standard  can  be  found  in  [178]. 
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procedure  ParallelGRASP(C',  m,  maxiter) 

1 BestSol  = oo; 

2 GenerateRandomNumberSeeds(si , S2, . . . 

3 do  (in  parallel)  i = 1, . . . , m — )• 

4 GRASP(C,  Pj,  Sj,  o6_)j, max  iter); 

5 if  obji  < BestSol  — >■ 

6 UpdateSol(o6ji,  Pi,  BestSol,  BestPerm); 

7 fi; 

8 od; 

9 return(BestSol,  BestPerm); 
end  ParallelGRASP  ; 


Figure  6.5:  Pseudo-code  for  parallel  GRASP 

In  parallelizing  GRASP  for  solving  the  MAP  the  key  observation  is  that  GRASP 
is  inherently  parallel  due  to  its  randomized  nature  [153].  That  is,  each  iteration  of 
the  algorithm  is  completely  independent  from  the  other  iterations,  thereby  two  or 
more  iterations  can  be  performed  simultaneously  without  the  need  to  communicate 
with  each  other  since  no  information  needs  to  be  shared.  This  feature  perfectly 
matches  the  message  passing  parallel  computational  model,  where  it  is  advan- 
tageous to  have  as  little  communication  between  the  processors  as  possible.  The 
parallel  implementation  of  GRASP  for  solving  the  MAP  using  m processors  in  par- 
allel, is  given  in  pseudocode  in  Figure  6.5.  In  lines  2 and  3,  the  best  solution  found 
is  initialized  to  a high  value,  and  m random  number  seeds  whose  corresponding 
random  number  sequences  will  be  disjoint  are  created.  These  seeds  sj,  S2>  • • • , Sm 
and  the  instance  data  C which  is  a multidimensional  array  are  distributed  among 
the  m processors,  which  return  their  solution  with  the  corresponding  permutation 
matrix  in  obji,  Pi-  The  best  solution  found  among  the  processors  is  determined  in 
lines  5-7,  and  returned  as  the  final  solution  in  line  9.  Due  to  the  inherent  parallel 
nature  of  GRASP  it  is  guaranteed  that  the  speedup  will  be  almost  linear,  which  is 
clearly  illustrated  in  [153].  Note  that  there  is  also  an  alternative  way  to  parallelize 
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the  GRASP  heuristic,  where  instead  of  distributing  the  number  of  iterations  to 
be  performed  in  different  processors,  each  iteration  could  be  parallelized  to  utilize 
more  than  one  processors  working  in  parallel.  However  this  would  not  be  advanta- 
geous, since  each  GRASP  iteration  for  the  MAP  requires  a small  amount  of  CPU 
time  as  it  will  be  shown  in  §6.6.  Moreover,  by  parallelizing  each  iteration,  linear 
speedup  would  not  be  guaranteed. 

6.6  Experimental  Results 

Test  problems  were  constructed  by  randomly  placing  sensors  on  a given  2-D 
grid  at  height  500  meters  and  targets  on  another  2-D  grid  at  height  zero.  The  size 
of  the  target  grid  was  made  small  so  that  the  targets  would  be  very  close  to  one 
another.  Spherical  measurements  from  each  sensor  to  each  target  were  made  and 
Gaussian  noise  with  covariance  Rsm  was  added  to  each  measurement.  Additionally, 
false  alarms  were  generated  at  a rate  of  A = 5 per  square  kilometer  and  randomly 
placed  on  the  target  grid.  To  account  for  missed  detections,  approximately  MJPq 
measurements  were  randomly  deleted. 

For  constructing  the  test  instances,  various  values  for  M and  J were  used  and 
the  sensor  error  covariance,  probability  of  detection  and  probability  of  false  alarm  ^ 
were  all  varied  as  shown  in  Table  6.1.  All  three  factors  used  the  same  value  for 
all  sensors  m = 1,2,...,  M.  A sample  of  the  resulting  test  problems  are  listed  in 
Table  6.2.  Columns  4,  5 and  6 contain  the  factor  level  used,  columns  7 and  8 show 
the  actual  number  of  false  alarms  and  missed  detections  respectively,  and  the  last 
column  contains  the  size  of  the  cost  vector  c,  which  can  be  considered  as  the  size 
of  the  input  data. 

'^In  reality,  the  sensor  field  of  view  was  varied  which  in  turn  varied  the  probability  of  false 
alarm  since  the  false  alarm  rate  was  fixed  at  A = 5 per  square  kilometer. 
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In  Table  6.3  the  performance  of  GRASP  in  solving  the  instances  of  Table  6.2  is 
shown.  The  GRASP  algorithm  was  coded  in  FORTRAN  77,  and  tested  on  a SUN 
SPARCstation  IPX.  For  each  test  run,  the  random  number  generator  seed  was  set 
to  270001,  the  parameter  a = 0.1  (see  §6.4.3),  and  the  maximum  number  of  iter- 
ations was  set  to  100.  Columns  2 through  4 in  Table  6.3  contain  respectively,  the 
best  solution  found  by  GRASP,  the  CPU  time  needed  to  complete  100  iterations, 
and  the  iteration  number  where  the  best  solution  was  attained.  The  last  column 
contains  the  cost  of  the  true  partition  of  each  instance.  In  all  of  the  instances 
GRASP  found  a lower  cost  solution,  than  that  of  the  true  partition.  This  is  to 
be  expected  since  the  maximum  likelihood  cost  function  is  a probabilistic  function 
which  is  subject  to  variability.  However  although  the  most  likely  partition  might 
not  (possibly  not)  correspond  to  the  optimal  partition,  the  optimal  partition  pro- 
vides a very  good  estimate  of  the  true  partition.  This  is  illustrated  for  example 
in  the  partitioning  provided  by  the  solution  of  GRASP  for  the  instance  Inst03 
as  illustrated  in  Figure  6.6.  In  InstOS  we  have  5 sensors  observing  5 targets,  and 
Zij  corresponds  to  the  measurement  of  sensor  i for  some  target  j.  For  target  5 
for  example  we  can  see  that  the  true  partition  should  be  {zi5,  Z35,  Z55,  Z45)  but  the 
partition  (zi5,  Z35,  Z55,  Z4e)  obviously  has  a lower  cost. 

The  results  presented  in  this  chapter  are  part  of  an  ongoing  research  in  the 
development  of  efficient  algorithms  for  solving  difficult  multidimensional  assign- 
ment problems  in  the  context  of  target  tracking.  Current  research  is  performed  in 
modifying  the  existing  data  structures  for  solving  large-scale  sparse  MAPs. 

6.7  Three  Dimensional  Assignment  Problem 

In  this  section  preliminary  results  regarding  a GRASP  implementation  for  solv- 
ing the  3-dimensional  assignment  problem  are  presented.  The  3-dimensional  as- 
signment problem  (3- DAP),  or  the  (axial)  3-dimensional  assignment  problem  as  it 
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Table  6.1:  Test  Problems  and  Factor  Levels 


Factor 

Level 

Value 

Sensor  Error  Covariance,  Rm,im 

1 

diag{0.25m  0.25mrad  0.25mrad} 

2 

diag{  1.00m  l.OOmrad  l.OOmrad} 

Probability  of  Detection, 

1 

0.95 

2 

0.90 

3 

0.70 

Probability  of  False  Alarm,  Pj^ 

1 

0.0200 

2 

0.2615 

3 

0.9361 

Table  6.2:  Test  Problem  Instances 


Instance 

j 

M 

factor  level 

qty  of 
false  alms 

qty  of 
miss  det 

P 

Pfa 

POm 

c| 

InstOl 

3 

5 

1 

1 

2 

2 

2 

900 

Inst02 

3 

5 

1 

1 

2 

2 

2 

960 

InstOS 

5 

5 

1 

1 

2 

2 

2 

7560 

Inst04 

5 

3 

1 

1 

2 

2 

2 

210 

InstOS 

10 

3 

1 

1 

2 

2 

2 

1320 

InstOG 

10 

3 

1 

1 

2 

2 

2 

1331 

Inst07 

20 

3 

1 

1 

2 

2 

2 

9261 

Table  6.3:  GRASP  computational  results 


Instance 

GRASP  Sol. 

CPU  time 

iterations 

True  Partition 

InstOl 

-52.2 

0.465 

1 

-52.1 

Inst02 

-63.5 

0.476 

1 

-58.0 

InstOS 

-100.5 

3.64 

6 

-48.5 

Inst04 

-44.5 

0.16 

3 

-36.0 

InstOG 

-126.7 

2.21 

2 

-101.8 

InstOG 

-131.7 

2.19 

1 

-107.1 

Inst07 

-288.6 

18.69 

1 

-163.9 
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Figure  6.6;  InstOS  GRASP  Partitioning 

is  also  called,  was  first  stated  by  Pierskalla  [159]  as  a straightforward  extension  of 
the  classical  two-dimensional  assignment  problem.  The  problem  formulation  with 
a brief  introduction  is  given  in  §2.7.2.  The  permutation  formulation  of  (2.38)  will 
be  used  throughout  this  section. 

Among  the  earlier  articles  published  on  the  3-dimensional  assignment  problem 
Pierskalla  [160]  is  the  first  to  make  an  in-depth  description  of  the  3-DAP,  showing 
that  most  of  the  results  reached  for  the  3-DAP  can  be  extended  to  the  multidi- 
mensional assignment  problem  (MAP).  In  Frieze  [74]  a slightly  generalized  form  of 
the  3-DAP  is  considered.  An  equivalent  bilinear  formulation  of  the  3- DAP  is  given 
and  a simple  way  of  finding  good  but  by  no  means  necessarily  optimal  solutions  is 
described.  This  is  probably  the  first  proper  attempt  of  attacking  large  problems 
where  integer  programming  and  branch  and  bound  methods  cannot  be  used.  The 
3-DAP  is  shown  to  be  an  A/"P-hard  problem  in  Garey  and  Johnson  [78]  who  in 
turn  base  their  work  on  Karp  [111]. 
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Since  many  algorithms  derived  for  the  3-DAP  are  of  branch  and  cut  type  which 
use  the  facial  structure  of  the  MAP,  this  has  been  the  subject  of  study  in  sev- 
eral articles.  Balas  and  Saltzman  [15]  examine  the  the  facial  structure  of  the 
3-dimensional  assignment  polytope  (i.e.  the  convex  hull  of  feasible  solutions  to 
the  problem)  and  identify  two  facets  belonging  to  the  polytope.  O(n^)  separation 
algorithms  are  given  for  these.  Balas  and  Qi  [14]  continue  the  above  work  and 
present  an  O(n^)  separation  algorithms  for  earlier  identified  classes  of  facets  as 
well  as  for  a new  class  (notice  that  since  there  are  variables  these  algorithms 
are  linear-time).  Qi,  Balas  and  Gwan  [164]  base  their  work  upon  linear-time  sep- 
aration algorithms  for  different  facets  and  construct  a polyhedral  procedure  for 
solving  the  3-DAP. 

In  this  section,  we  present  a greedy  randomized  adaptive  search  procedure 
(GRASP)  implementations  for  the  3-DAP.  Computational  results  for  the  proposed 
GRASP  are  given  for  instances  of  the  3-DAP  considered  by  Balas  and  Saltz- 
man [16],  Grama  and  Spieksma  [54],  as  well  as  Burkard,  Rudolf  and  Woeginger  [42]. 

The  remainder  of  this  section  is  organized  as  follows.  In  §6.7.1  we  describe  the 
GRASP  for  the  3-DAP.  Experimental  results  are  presented  in  §6.7.2,  and  conclud- 
ing remarks  are  made  in  §6.7.3. 

6.7.1  A GRASP  for  the  3-DAP 

A general  description  of  the  GRASP  heuristic  is  presented  in  §2.10.7.  In  the 
remainder  of  this  section  a GRASP  for  the  3-DAP  will  be  described.  In  §6.7.1  the 
first  of  the  two  GRASP  stages,  the  construction  phase,  is  presented.  The  second 
GRASP  stage,  the  local  search  is  described  in  §6.7.1. 
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Construction  phase 

In  the  GRASP  construction  phase  the  costs  are  to  be  sorted  and  a restricted 
candidate  list  (RCL)  created  which  is  updated  each  time  an  assignment  is  made, 
to  include  only  permissible  assignments.  Initially  the  RCL  is  constructed  by  ar- 
ranging the  Cjjjt’s  in  ascending  order,  and  taking  only  the  * /3  smallest  (where 
(3  is  a parameter).  This  is  performed  in  lines  1 and  2 of  the  pseudo-code  dis- 
played in  Figure  6.7.1.  Let  us  define  a partial  solution  to  the  problem  with  the  set 
r = A:i), . . . , {iriiri  kr)},  which  means  that  r assignments  have  been  made 

corresponding  to  the  partial  permutations 


pUi)  - k,  q(ki)  = ii  for  l = 


Upon  making  the  assignment,  an  element  is  selected  at  random  from  the  first 
a * (size  of  the  RCL)  elements  of  the  RCL,  and  is  added  to  the  partial  solution 
(lines  4 and  5 in  Figure  6.7.1).  Then  in  line  6 the  RCL  is  updated  to  reflect  the 
assignment,  by  arranging  the  costs  that  correspond  to  permissible  assignments 
represented  here  by  the  set  C = {c^jt  \ i ^ ^ ^ kr}  in  ascending  order, 

and  including  only  the  | C | */?  smallest  in  the  RCL.  Finally  in  line  8 of  Figure  6.7.1 
the  last  assignment  is  made,  which  is  the  only  permissible  one. 


Local  search  phase 

In  the  local  search  procedure  the  current  solution  is  improved  by  searching  its 
neighborhood  to  see  if  a better  solution  is  found.  If  so,  the  solution  is  updated 
and  a new  neighborhood  search  is  initialized.  This  means  that  the  definition  of 
the  neighborhood  N{s)  is  crucial  to  the  performance  of  the  local  search. 
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procedure  Construct(costArray,  solution) 

1 SortCosts(costArray); 

2 createRCL; 

3 do  r = 1, . . . , n-1  ^ 

4 A:r)=SelectRandomElement  (RCL) ; 

5 r = ru 

6 UpdateRCL(costArray , ^r)  ,RCL) ; 

7 od; 

8 r = ru  A:„); 

9 return(solution); 
end  Construct; 


Figure  6.7:  GRASP  construction  stage  pseudo-code 

In  our  case  a solution  corresponds  to  a pair  of  permutations  {p,q)  and  the 
solution  space  of  the  3-DAP  therefore  consists  of  all  (n!)^  possible  combinations  of 
permutations. 

Let  us  first  define  the  difference  between  two  permutations  p and  p„etu  to  be 


KP^Pnew)  = {i  I P{i)  # Pnewii)}, 


and  the  distance  between  them  is 


^{PiPnew^  I ^iPiPnew)  |- 

A neighborhood  can  of  course  be  defined  in  many  different  ways  for  different  prob- 
lems, but  for  the  GRASP  local  search  a A:-exchange  neighborhood  is  used.  A 
/c-exchange  neighborhood  is  defined  to  be 

Nk{p)  = {p  new  I ^{PiPnew)  ^ 


for  any  2 < k < n.  In  this  implementation  we  have  chosen  to  use  k—2,  a 2- 
exchange  neighborhood.  When  defining  the  neighborhood  of  a solution  (p,  q)  in 
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terms  of  2-exchanges,  the  natural  choice  is  of  course  to  let  it  be  all  possible  2- 
exchange  permutations  belonging  to  (p,  q).  However  the  size  of  that  neighborhood 
is  (2)  , which  is  very  large  even  for  small  values  of  n.  Therefore,  the  neighborhood 
of  a solution  (p,  q)  is  here  defined  to  be  all  2-exchange  permutations  of  p plus 
all  2-exchange  permutations  of  q.  This  means  that  for  a solution,  that  is,  two 
permutations  p,q  G tt^v,  the  2-exchange  neighborhood  is  defined  to  be 


^2{Pi  9)  •^2(p)  U -^2(9)  \jpnexui  Qnew  \ ^(P)Pneu;)  T Q.new)  — • 


Hence,  the  size  of  the  neighborhood  is  | A^2(p)  | + | -^2(9)  I = (2)  + (2)- 
procedure  LocalSearch  each  of  the  neighbor’s  cost  is  compared  with  the  cost  of 
the  current  solution.  If  the  cost  of  the  neighbor  is  lower  the  solution  is  updated,  the 
search  halted  and  a search  of  the  new  neighborhood  initialized.  More  specifically 
it  is  done  like  this:  For  each  solution  2-exchange  permutations  are  created  for  p 
and  q (line  8 of  the  pseudo-code  displayed  in  Figure  6.7.1).  If  neither  of  them 
improves  the  solution,  the  search  continues  until  a better  solution  is  found  or  the 
neighborhood  is  completely  searched  (lines  6 through  18).  When  a neighbor  with 
lower  cost  is  found  the  solution  is  updated  and  a new  search  begins  (lines  9 through 
15).  Finally  the  procedure  returns  the  best  found  solution. 


6.7.2  Computational  Results 

Our  GRASP  for  the  3-dimensional  assignment  problem  G3APS,  was  imple- 
mented in  Fortran  77  and  tested  on  a SUN  SPARCstation  IPX.  In  this  section 
we  will  present  results  from  test  runs  on  problems  considered  by  Balas  and  Saltz- 
man  [16],  Grama  and  Spieksma  [54],  as  well  as  Burkard,  Rudolf  and  Woeginger 
[42].  The  problems  supplied  by  Grama  and  Spieksma  [54]  and  Burkard,  Rudolf 
and  Woeginger  [42]  both  considered  cases  where  the  cost  coefficients  could  be  de- 
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procedure  LocalSearch(solution); 

1 QUIT=false; 

2 while  not  QUIT  do-> 

3 if  [search  finished]  QUIT=true; 

4 STOP=false; 

5 counter=0; 

6 while  not  STOP  do— 

7 counter=counter+l; 

8 ProducePermutation(counter ,p,q,pnew,qnew) ; 

9 if LowerCostCp, counter, pnew)  -4 

10  p=pnew; 

11  ST0P=true; 

12  else  if  LowerCostCq, counter, qnew) 

13  q=qnew; 

14  ST0P=true; 

15  fi; 

16  fi; 

17  if  (not  STOP)  and  (counter=neighbrhdsize)  QUIT=true; 

18  od; 

19  od; 

20  return(solution) ; 
end  LocalSearch; 


Figure  6.8:  GRASP  local  search  stage  pseudo-code 
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composed,  although  in  two  different  ways,  making  the  problem  instances  easier  to 
approximate.  For  further  information  on  the  different  problem  instances  see  [16], 
[54]  and  [42]  respectively. 

The  results  of  the  test  runs  are  shown  in  tables  6.4,  6.5  and  6.6.  The  ran- 
dom number  seed  used  was  270001.  The  first  columns  of  the  tables  tells  which 
problem  was  considered,  its  dimension  (n),  the  values  of  RCL  parameters  a and 
/?  and  the  number  of  GRASP  iterations  taken.  The  next  two  columns  display  the 
results  reached  by  the  proposed  GRASP  in  terms  of  CPU-seconds  and  objective 
function  value.  The  following  columns  show  the  objective  function  values  and  in 
some  cases  the  CPU-seconds  reached  by  the  different  heuristics  presented  in  Balas 
et  al.  [16],  Grama  et  al.  [54]  and  Burkard  et  al.  [42]  respectively.  In  table  6.4  the 
last  four  columns  show  the  objective  function  values  reached  in  Balas  et  al.  [16] 
for  the  different  heuristics  greedy  (z^),  maxregret  (z^)  and  maxregret  followed  by 
variable  depth  interchange  (z^)  as  well  as  the  optimal  value  (z*).  In  table  6.5  the 
last  four  columns  show  the  results  reached  in  Grama  et  al.  [54]  for  three  approxi- 
mation algorithms  {cjj^cik  and  cjk)  as  well  and  a lower  bound.  Finally  the  last 
seven  columns  of  table  6.6,  display  the  results  reached  in  Burkard  et  al.  [42]  for 
the  different  heuristics  maxregret_2,  simple_3,  Ish  and  simpleJsh.  (Only  the  ob- 
jective values  are  listed  for  simple_3,  since  the  running  times  in  CPU-seconds  were 
negligible.) 

Discussion  of  the  results 

As  seen  in  tables  6.4,  6.5  and  6.6,  the  performance  of  GRASP  is  based  upon 
the  instance  solved  and  the  parameters  used.  However,  in  almost  every  case  the 
solution  reached  provided  a good  approximation  of  the  optimal  solution  or  the  best 
known  solution. 
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Table  6.4:  Statistics  summarizing  test  runs  on  the  data  by  Balas  & Saltzman 


solutions  reached  in  [16] 


n 

a 

0 

iter 

CPU-seconds 

value 

z9 

zh 

2* 

opt  value 

4 

0.5 

0.10 

32 

0.220 

43.4 

53.6 

52.6 

43.2 

42.2 

4 

0.5 

0.10 

64 

0.090 

43.4 

53.6 

52.6 

43.2 

42.2 

6 

0.5 

0.10 

32 

0.456 

40.6 

90.2 

76.0 

45.4 

40.2 

6 

0.5 

0.10 

64 

0.426 

40.2 

90.2 

76.0 

45.4 

40.2 

8 

0.5 

0.10 

32 

1.582 

39.0 

81.4 

59.6 

33.6 

23.8 

8 

0.5 

0.10 

64 

1.456 

36.6 

81.4 

59.6 

33.6 

23.8 

10 

0.5 

0.10 

32 

4.424 

47.8 

84.4 

50.8 

40.8 

19.0 

10 

0.5 

0.10 

64 

3.870 

42.6 

84.4 

50.8 

40.8 

19.0 

12 

0.5 

0.10 

32 

9.900 

47.8 

87.0 

40.2 

24.0 

15.6 

14 

0.5 

0.10 

64 

16.782 

50.8 

86.4 

64.0 

22.4 

10.0 

16 

0.5 

0.10 

64 

31.882 

53.4 

78.2 

58.8 

25.0 

10.0 

18 

0.5 

0.10 

64 

54.854 

60.0 

62.4 

21.8 

17.6 

6.4 

20 

0.5 

0.10 

64 

88.208 

66.6 

77.4 

75.8 

27.4 

4.8 

22 

0.5 

0.10 

64 

136.454 

77.4 

93.4 

47.8 

18.8 

4.0 

24 

0.5 

0.10 

64 

203.770 

65.2 

91.0 

59.1 

14.0 

1.8 

26 

0.5 

0.10 

64 

298.133 

83.0 

107.3 

36.0 

15.7 

1.3 

Table  6.5:  Statistics  summarizing  test  runs  on  the  data  by  Crama  & Spieksma 


solutions  reached  in  [54] 


problem 

n 

Q 

0 

iter 

CPU-seconds 

value 

CIJ 

CIK 

CJK 

lower  bound 

3da99nl 

33 

0.5 

0.25 

32 

580.48 

1640 

1618 

1636 

1655 

1607 

3da99n2 

33 

0.5 

0.25 

32 

567.19 

1421 

1416 

1416 

1411 

1395 

3da99n3 

33 

0.5 

0.25 

32 

566.33 

1620 

1609 

1633 

1632 

1604 

3dal98nl 

66 

0.4 

0.25 

32 

13816.3 

2778 

2692 

2687 

2668 

2654 

3dal98n2 

66 

0.4 

0.25 

32 

13769.4 

2535 

2500 

2481 

2469 

2433 

3dal98n3 

66 

0.4 

0.25 

32 

13775.4 

2863 

2813 

2775 

2791 

2748 

3dij99nl 

33 

0.5 

0.25 

32 

570.53 

4823 

4883 

4861 

4875 

4772 

3dij99n2 

33 

0.5 

0.25 

32 

568.72 

5092 

5172 

5142 

5149 

5035 

3dij99n3 

33 

0.5 

0.25 

32 

569.52 

4306 

4352 

4383 

4401 

4260 

3dil98nl 

66 

0.4 

0.25 

32 

14020.2 

9740 

9809 

9780 

9915 

9633 

3dil98n2 

66 

0.4 

0.25 

32 

14153.7 

9001 

9147 

9160 

9142 

8831 

3dil98n3 

66 

0.4 

0.25 

32 

14054.5 

9815 

9939 

9888 

9957 

9670 

3dl299nl 

33 

0.5 

0.25 

32 

547.16 

143 

140 

135 

136 

133 

3dl299n2 

33 

0.5 

0.25 

32 

550.05 

143 

141 

137 

139 

130 

3dl299n3 

33 

0.5 

0.25 

32 

548.34 

141 

135 

136 

137 

130 

3dll98nl 

66 

0.4 

0.25 

32 

14400.3 

306 

295 

293 

296 

283 

3dll98n2 

66 

0.4 

0.25 

32 

13899.7 

309 

294 

298 

294 

281 

3dll98n3 

66 

0.4 

0.25 

32 

13841.4 

309 

295 

296 

293 

280 

Table  6.6:  Statistics  summarizing  test  runs  on  the  data  by  Burkard,  Rudolf  and 
Woeginger 

solutions  reached  in  [42] 

maxreg_2  sim_3  Ish  simJsh 


n 

iter 

seconds 

value 

seconds 

value 

value 

seconds 

value 

seconds 

value 

4 

32 

0.0478 

444.31 

0.007 

621.79 

468.17 

0.003 

443.69 

0.007 

443.70 

6 

32 

0.2549 

634.68 

0.036 

946.27 

661.97 

0.011 

634.46 

0.009 

634.20 

8 

32 

0.8527 

820.12 

0.086 

1289.20 

871.02 

0.019 

820.26 

0.016 

819.94 

10 

32 

2.2650 

961.35 

0.199 

1569.90 

1015.79 

0.025 

960.48 

0.024 

960.55 

12 

32 

4.8715 

1188.67 

0.394 

2071.26 

1256.68 

0.031 

1188.28 

0.024 

1188.02 

14 

32 

9.7192 

1471.07 

0.697 

2630.59 

1565.55 

0.033 

1469.19 

0.031 

1469.27 

16 

32 

17.8941 

1479.37 

1.162 

2788.21 

1576.04 

0.041 

1476.80 

0.036 

1476.99 
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Among  the  problems  tested  it  was  on  the  problems  originating  from  Balas  and 
Saltzman  [16]  our  GRASP  performed  most  poorly.  Only  in  one  case  did  it  find 
the  optimal  solution  (for  n=6,  o;=0.5,  /?=0.1  and  64  iterations).  The  structure  of 
the  problems  clearly  favors  the  maxregret  followed  by  variable  depth  interchange 
heuristic  presented  in  Balas  et  al.  [16]. 

On  the  problems  supplied  by  Grama  and  Spieksma  [54]  on  the  other  hand 
G3APS  performance  was  very  good.  For  every  problem  of  type  II  it  produced 
solutions  very  close  to  the  lower  bounds  outperforming  the  approximation  algo- 
rithms presented  in  Grama  et  al.  [54]  and  for  the  problems  of  type  I its  solutions 
were  almost  as  good.  Only  for  the  problems  of  type  III  were  the  approximation 
algorithms  decidedly  better,  but  still  G3APS  gave  good  upper  bounds  of  the  opti- 
mal solution.  The  computational  effort  seems  quite  large,  especially  when  n=66, 
but  there  is  no  data  to  compare  with. 

Also  for  the  problems  provided  by  Burkard,  Rudolf  and  Woeginger  the  com- 
putational effort  was  rather  heavy.  Even  though  G3APS  gave  very  good  solutions 
compared  to  most  of  the  heuristics  presented  in  Burkard  et  al.  [42],  the  heuristic 
simpleJsh  must  be  considered  superior  for  these  problems.  While  the  CPU-time 
required  by  G3APS  quickly  increased  as  n grew  larger,  the  CPU-time  needed  by 
simpleJsh  only  grew  slowly  as  it  made  use  of  the  special  structure  of  the  problem. 

6.7.3  Concluding  Remarks 

The  GRASP  for  the  3-DAP  presented  in  this  section  has  been  shown  to  give 
very  good  upper  bounds  for  several  different  instances  of  the  3-DAP.  However  this 
is  often  done  at  the  cost  of  high  running  times.  There  are  however  many  ways 
to  improve  its  efficiency.  A further  refinement  of  the  implementation  to  handle 
sparse  3-DAP  instances,  and  parallelization  of  the  algorithm  a areas  for  further 
improvement.  Even  though  other  heuristics  are  more  efficient  for  special  cases  of 
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the  3-DAP,  we  have  not  seen  any  other  heuristic  providing  good  solutions  for  such 
a wide  variety  of  problems.  This  is  of  course  partly  related  to  the  relatively  small 
amount  of  work  done  on  the  3-DAP. 
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